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FOREWORD 


This document, though an official release of the Apollo Program Office, is furnished 
for information purposes only. Its purpose is to create awareness, stimulate interest 
and further promote understanding in the art and science of making real-life forecasts 
and their subsequent utilization in the control of space vehicle weight and performance 
throughout the Apollo Program. 

This book is primarily intended for those in the Apollo Program who are responsible 
for the administration, design, development, manufacture, and test of the Apollo Sys- 
tem. New theorems have been developed, as well as application of proven techniques 
but more importantly, a weight/performance forecasting methodology has been devel- 
oped and auto mat ed - The text emphasizes the utilization of forecasting devices as ap- 
plied to space vehicle weight and performance since these two parameters are of vital 
interest to all levels of management as well as technical personnel. Further, weight 
is tangible and readily measurable and can be readily related to performance . 

The text provides , to those who wish to apply the developed methodology , all details 
necessary to do so and includes the mathematical development, computer program 
user's manuals and necessary instructions and procedures. 

Forecasts and Appraisals for Management Evaluation text is intended to be a construc- 
tive aid to the NASA Apollo team in assisting them in the weight and performance area. 

- — - 

Samuel C . Phillips " 
Major General, USAF 
Director, Apollo Program 
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PREFACE 


This book provides an insight into the problems associated with the development of a 
system for maintaining effective control over large and complex aerospace programs. 
Chapters 1, 2, and 3 contain general considerations, concepts and the utilization of a 
general data-handling system with particular application to weight/performance control 
developed to assist managers and contractors on the Apollo Program. Possible appli- 
cations to cost and schedule monitoring, reliability, and other fields are briefly 
discussed. 


The specific application to weight and performance control, with attendant exhibits of 
work sheets, printouts, and reporting forms to management, is contained in Chapter 3. 
Additional elements of interest to the management evaluation task, along with the 
mathematical procedures and mechanics of the computational system, are appended 
for reference purposes in Book II. 

The primary objective in compiling this book is to offer, for general consideration and 
possible utilization, a complete and comprehensive system on Forecasts and Apprais- 
als for Management Evaluation useful in the management of highly complex, increas- 
ingly sophisticated, interrelated programs . 


CHAPTER 1 


AN INTRODUCTION TO FORECASTS AND APPRAISALS FOR 
MANAGEMENT EVALUATION 


1.1 BACKGROUND 

Since the end of World War II there has been a growing need for management informa- 
tion systems which can process a mass of technical data from many sources, assess 
the significance of the distilled information and present clear, concise facts to manage- 
ment. Today we are in a period in which mathematics and decision logic are coupled 
to provide management with quantitative evaluations of what previously had been purely 
subjective considerations. The outcome of this union has been the introduction of 
several new systems for supplying management information. This book describes one 
of these new systems, called Forecasts and Appraisals for Management Evaluation 
(Acronym FAME) . This system develops forecasts of program status providing man- 
agement with the increased perception needed to maintain control of large and complex 
technical programs. 

FAME is a process which assesses the facts of yesterday and the actualities of today, 
and forecasts the probable events of the future. The process provides quantitative 
values for stated problems, defines their magnitude, and describes the effects of 
alternative actions or inaction by management. FAME can be applied to data of a 
historical nature or to information having inherently variable characteristics . The 
system is dynamic . It is capable of responding to changes in design, shifts of em- 
phasis , and determining the program impact of such changes . It provides the mean s 
for giving quantitative values- to information which otherwise would be limited to sub- 
jective evaluation by management. 

The requirement for FAME was spawned by the need for a dynamic management tool 
which would accurately forecast the weight and performance of Space Vehicles during 
all phases of program development. The principles of FAME are also applicable to 
cost, schedule, flight performance, ground facilities utilization and reliability evalua- 
tion and trade-offs among them. 

1.2 THE LOGIC BEHIND FAME 

In a broad sense, FAME may be characterized as being operations research, i.e. , 
research concerned with applying scientific method to the problems facing management. 
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Yet, as A. Kaufmann points out, operations research is not in itself a science but 
rather a scientific attitude toward management phenomena. Kaufmann says: 

"There are times when there is little difference of meaning 
between Econometrics and Operations Research since the 
borderlines between the economic and physical area of tech- 
nology and management are not cj.ea.ny ueiineu. 

Adding to these thoughts, one observes that a revolutionary period of mathematical and 
decision logic is upon us. This demands that advanced technologies and the science of 
management be welded together to provide quantitative predictions upon which qualita- 
tive management decisions may be based. Therefore, forecasts, as such, are the 
heart of FAME . 

It could be argued that FAME is a hybrid form of statistical analysis. This is not 
worth debating, for it is through statistical inferences as determined by probability 
theory that the marriage of mathematics and logic takes place. 

Accentuating the word "forecasting" draws attention to an area which concerns us most. 
Others use "trend" and "projection" in a similar vein. Throughout this book the words 
"forecast" and "prediction" are used interchangeably. The term "Prediction Analysis 
which appears frequently, refers to that one major aspect of the FAME system in 
which pre-trended data is subjected to mathematical analysis. 

1.2.1 REQUIRED ATTRIBUTES OF FAME 

Forecasting techniques to be useful must have the attributes of consistency, efficiency, 
and sufficiency. When applied to space vehicles these attributes must be stringently 
defined because the total number of observations available are limited and will not 
increase indefinitely, as normally assumed in the pure statistical sense. Accordingly, 
the following definitions are provided. 

1.2. 1.1 Consistency 

Consistency is that attribute of FAME which is distinguished by the convergence of the 
estimated parameter towards a final value each time an additional set of data is added 
to the initial set of observations. As the parameters converge the probability of pre- 
dicting a value, other than the one upon which they are converging, diminishes rapidly 
This is referred to as "targeting. " 

~ Numbers in parentheses refer to references in Section R of this book. 
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^ 1.2. 1.2 Efficiency 

Efficiency is that attribute of FAME which is distinguished by the var ian t of the esti- 
mated value toward a finite variance each time an additional set of data is added to the 
initial set of observations. It is further stipulated that this variance be less than or 
equal to the allowable variance in the final measured value. This is designated as 
"accuracy." 

1.2. 1.3 Sufficiency 

Sufficiency is that attribute of FAME which is distinguished by the extraction of all 
possible information from the observed sets of data. 

1.3 THE VALUE OF FORECASTS 

All decisions invarably involve predictions. For, if there were not a concern for the 
future a decision would not be required. Simply stated, a manager must be able to 
forecast the future, since today's decisions largely influence tomorrow's occurrences. 
It is the probable impact of these occurrences which dictate today's decisions. It is 
this concern with key program elements, such as program cost, schedule, and per- 
formance that prompts the introduction of new techniques to the fieldofprogram control. 

In any program subject to management control, budgets are established which desig- 
nate a sum of appropriated money which can be spent over a specific time for a desired 
output. This output could, as an example, be a number of production units. With 
goals and limits established, management usually institutes some type of reporting 
system in order to measure progress against these goals and limits. In principle, 
this procedure is a familiar and simple enough means for maintaining control. It 
amounts to taking a look at existing conditions and deter mining whether progress is 
acceptable. If the rate of spending suddenly rises, the expenditure rate may result in 
costs per production unit exceeding the sale price. If the trend in spending can be 
sensed at an early date, the problem may be solved by instituting cost reductions or 
increasing the sale price of the production unit. If the problem is recognized too late 
m the program, the result could be substantial losses in time, dollars, and performance. 

In many respects technical parameters can be viewed along similar lines. Technical 
budgets (control limits) are established which are compatible with the over-all tech- 
nical program requirements. These requirements are then further detailed into tech- 
nical requirements by specifications at each program tier. These specified controls 
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then become the technical budget. These budgets are not irrevocably fixed, that is 
there may be several other combinations of parameters which will be compatible with 
over-all program objectives. On a very complex engineering development program 
such as Apollo, these budgets are established using the latest "state-of-the-art" data 
and the highest caliber of technical personnel. In general, there are three supporting 
areas of engineering know-how: proven engineering facts, engineering calculations 
based on proven theory, and estimations of what can be accomplished based on knowl- 
edge obtained from previous programs. Hence, at the beginning of a program there is 
an uncertainty that the established technical requirements are near optimum or even 
that they are truly compatible, or that the work can be accomplished within the mone- 
tary and schedule constraints. 

1.3.1 THE IMPORTANCE OF TIMING 

As a program progresses, problems are recognized and requirements are adjusted by 
conducting trade-off studies. However, it is not enough for management to base re- 
quirement changes on current status. 

Consider the situation of the program manager who is given the program status charts 
shown in Figure 1-1. He is generally aware of a rising value of the particular item 
reported, but relies on assurances from many sources that the program is being 
brought under control. Consequently, no further action is taken. 

Next month, the shock arrives. The growth, illustrated in Figure 1-2, has not abated. 
The value is now over its control limit and continuing to climb. Corrective actions, 
which now must be made late in the program will result in inordinately large expendi- 
tures of time and money, with concurrent scrapping or reworking of hardware items. 
Delays are now to be anticipated, since reworked parts, must be requalified with 
schedule slippage in turn reflected in an ever widening wave throughout the program. 

The key ingredient supplied by FAME is the timely recognition of problems and de- 
termination of their magnitude such that program disturbances can be minimized. 

Figure 1-3 illustrates the impact of timing on the progress of a program which has an 
established control limit and a target (delivery) date. Line "A" represents current, 
on-going progress; dotted line "B" indicates a forecasted trend where recognition of 
problems and resultant management action are not taken until late in the program. 
Line "B" graphically illustrates the necessity for management to recognize problems 
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Status Index 



Figure 1-1. Typical Program Status as Seen at Month Eight 



Figure 1-2. Typical Program Status as Seen at Month Nine 


1-5 





Figure 1-3. Impact of Timing on Program Control 

early enough to economically take the necessary corrective actions. Line "C" indi- 
cates the results of early corrective action where managers have anticipated problems 

Actually, of course, real-life situations are of considerably greater complexity. Many 
independent contractors are engaged in design, production, test, and operation of a 
myriad of interdependent space vehicle components. 

1.4 ORGANIZATION AND OPERATION OF FAME 

1.4.1 PROGRAM CONTROL ELEMENTS 

Three organizational elements are of special interest in Program Control: 

a. Program Control Management. 

b. Systems Engineering. 

c. Design Agencies. 
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Management responsibilities include keeping a project within control limits; constant 
surveillance to assure that these limits provide for the best utilization of time and 
money, and the successful design and development of the end product. 

The ingredients necessary for program control include : 

a. Mission requirements. 

b. Baseline requirements. 

c. Program status (data and displays). 

Mission requirements establish the over-all objectives of the program which are of 
highest importance. The baseline requirements are the control limits for the myriad 
of constituent elements which go into the complex equipment being designed to meet 
mission requirements. For the Apollo Program, an operational mission requirement 
is the successful landing of Astronauts On the moon; and their safe return. Baseline 
requirements or control limits have been established for the individual space vehicle 
stages and modules, for example, for their weights and performance. Program status 
information supplies the necessary data on progress to permit program control to 
function. 

In analogy, program control is much like the operation of a room thermostat. The 
mission requirements are analogous to the desired temperature for comfort. Baseline 
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requirements correspond to the setting of 
the thermostat , as well as establishing the 
li mi ts of operation of the heating or cool- 
ing equipment. Operation of a heating and 
cooling system is analogous to the actions 
taken by the design agencies to produce 
the required result. Room temperature, 
like status data, is sensed and corrective 
actions are produced accordingly by the 
thermostat. So far, there is nothing radi- 
cally different from conventional program 
control techniques. 

1.4.2 THE TIME DIMENSION 

The analogy of program control to a thermostat changes, however, when a new dimen- 
sion, time, is added to this control system. On complex programs , such as Apollo, 
as well as in many similar areas of management concern, there is a recognized need 
to meet requirements not today but at some point in the future, usually at the time of 
launch — "the moment of truth." Because the design, tooling, production, anddevelop- 
ment cycle involves a period of many years, information is required by management 
which reflects not only present status , but also anticipated problems . By looking 
ahead, costly emergency actions and schedule slippages can be avoided. 

Incorporation of this dimension of future time in program control can be achieved with 
FAME. This system was developed to meet the expanding needs of Aerospace Pro- 
gram management. FAME has demonstrated its value for Apollo Program Control and 
appears to have potential application to numerous other programs. Today, FAME 
techniques are providing program management with valuable information of proven 
accuracy. 

1.4.3 ORGANIZATION OF FAME 

The organization and operation of FAME are simple in concept. Figure 1-4 is a 
schematic illustration of the total process . The three key organization elements : 
Design Agencies, Systems Engineering, and Program Control Management are shown 
at the top and bottom of this chart. The large, central box indicates the relationship 
and nature of the Forecast and Appraisal System and encompasses the several kinds 
of operations performed by FAME . 



1-8 





Figure 1-4. Basic Organization and Operation of FAME 
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The heart of this system is Forecast Analysis (shown in Figure 1-5) which anticipates 
the expected health of the program, using observed status and control information to 
produce the forecasts necessary for program appraisal. The appraisal calculations 
translate these forecasts into meaningful indices which pinpoint key problem areas as 
well as indicate the significance of problem solutions. Both actual and potential prob- 
lems are identified. The criticality of suggested corrective action is noted for each 



Figure 1-5. Forecast Analyses Shown as the Heart of FAME, Using the 
Schematic Diagram of Figure 1-4 as a Basic Reference 

tier of concern, from the smallest component to the total space vehicle itself. In addi- 
tion, key, program-wide interrelationships are noted so that inter-program trade-offs 
can be made by program management when required to correct the problems noted. 
The FAME system provides total program control through application of statistical 
Forecasting techniques, with details reported on a regular (monthly) basis to keep 
management appraised of progress and notified of key potential problems. The 
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essential steps of FAME are described in simplified form in the following sections and 
•• in detail in the following chapters. 

1.4.4 REQUIREMENTS 

Essential to the operation of Forecast Analysis are clearly defined control limits, for 
without a firm basis for making comparisons or measuring progress , there can be no 
evaluation of progress. Mission requirements are established at the inception of the 
program and updated periodically but only after careful evaluation of the consequences 
of such changes. The mission requirements, reduced to specific baseline require- 
ments, are displayed as the starting point of a history control chart. On Figure 1-6 
two kinds of requirements are introduced: the control limit or ceiling for growth and 
the schedule dates of importance. The control requirements can be depicted in sim- 
plest form as two boundaries on the y and x axis respectively. 



Figure 1-6 . Measurable Requirements Reflected in Forecast Analysis Riot 
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1.4.5 DATA INPUT 

Status report information is the next ingredient required. Frequently, data is gen- 
erated by contractors and passed upward through many levels of management. Since 
such data are used as measures of contractor performance, each level in the manage- 
ment scale may be apprehensive about data going to higher management and a possible 
reluctance to supplying data may result. To overcome this reluctance, assurance 
about the intended use of the data is essential. Confidence can be generated by the 
feet that the output from the data is open for inspection and investigation by all con- 
cerned. The principal motivation for supplying data, however, is the data flow docu- 
mentation requirements. For the Apollo Program this requirement was established 
for weight/performance control by the "Weight and Performance Data Submittal Direc- 
tive. " The data supplied monthly shows observed values of weight and related data 
and would be displayed on a weight-time control chart, such as illustrated in Fig- 
ure 1-7. For ease of reference, data points in Figure 1-7 are connected by a line, 
and absolute values are printed alongside the plotted curve versus months. 

1.4.6 DATA PRE-PROCESSING AND FORECAST ANALYSIS 

Even though data provided by design agencies are accurate and authentic, changes in 
ground rules frequently occur and result in data presented on an inconsistent basis . 

To insure that data are being viewed on a consistent basis, the data is treated by a 
normalization process to remove the influence of non- random changes (delineated in 
paragraph 4.4). Following this the Forecast Analysis calculations are performed. 

This involves assessing inherent growth trends of the normalized, unbiased data and 
then solving mathematically for a forecast line which is extended to the shipping or 
launch date. 

This process is graphically shown on Figure 1-8. In this example, the forecast line 
exceeds the control limit, upper limit, indicating that management action is necessary. 

Predicted excesses or deficiencies to be removed are termed "buyoff' since this re- 
duction is presumably purchased at some cost in dollars, scheduling or reduced tech- 
nical performance. 

1.4.7 APPRAISAL 

Calculations are then performed to express this buyoff in terms of over-all program 
parameter numbers, designated as "deficiency. " The seriousness of the excesses 
over the control limit are assessed and assigned a "criticality" or "decision relevancy 
rating which in effect indicates the impact such deficiencies may have on the program. 
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Time t Ship 

Date 

Figure 1-7. Measurable Status Reflected in Forecast Analysis Plot 

The extracted information, together with any action recommendation, is transmitted 
formally to management via FAME reports which are issued regularly. A typical re- 
port currently being submitted for Weight/ Performance is the Forecast and Appraisal 
Status Transmittals conveniently named "FAST". 

1.4.8 MANAGEMENT DECISIONS 

The most apparent management decisions to be made from these FAST (Forecast and 
Appraisal Status Transmittals) reports and graphically shown on Figure 1-9 are: 

a. To make the necessary buyoff at the earliest consistent date, or, 

b. To raise the control limit to allow for the anticipated growth utilizing 
trade-offs in other system parameters. 
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Figure 1-8. Forecast Analysis Plot Showing Forecast Line and Buyoff 

While either of these, individually or in combination, is possible, the first manage- 
ment action customarily consists of providing for the conduct of studies to define the 
best method for achieving the indicated buyoffs. At this point the process reverts to 
the design agencies who study and with program management implement the proper 
change as indicated by such studies. The Forecast Analysis operation thereupon re- 
turns to its original surveillance status. 

Final judgment about the rate of program progress and any actions to be taken are 
made by management. To expedite the making of decisions, a degree of criticality 
is estimated for each problem identified by the analysis. 
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Figure 1-9. Program Management Decision 


The criticality notation pinpoints mission weight and performance weaknesses and aids 
in assessing, on a consistent basis, the magnitude of the problems likely to be en- 
countered in meeting weight and performance design criteria. The four degrees of 
criticality may be defined as follows: 

a. Critical - High probability that forecasted deficiencies wiU seriously 
impede successful accomplishment within control limit and requires 
major program decisions. 

b. Major weakness - Substantial probability that forecasted deficiencies wiU 
impede successful accomplishment within control limit. This alerts 
management to a potential problem due to a deficiency being higher than 
normally acceptable associated with this limitation. 
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c. Minor weakness - Probability that forecasted deficiencies are acceptable 
and c ons i stenlkwith similar deficiencies that were successful in the past. • 
No action or decision is required. 

d. Good shape - Excellent probability that forecasted values will continue 
below control limit. 

The outputs of Forecast Analysis, in short, can be compared to a health chart which 
shows current status and forecasts the health of the program at launch time . 

1.5 detailed description of the forecast and appraisal system 

Specific elements which are included in the routine exercised by the FAME System are 
listed in Table 1-1. 

Table 1-1 

Elements of Forecast and Appraisal System 


Data Pre-Processing 

a. Assessment of Mission Requirements, Trade-off Factor Extraction 

b. Change Analysis (Identification of Non- Random Changes) 

c. Normalization (Removal of Non-Random Changes) 

d. Time Transformation (when required) 

Forecast Analysis 

e. Trend Analysis Using Observed Estimated/Calculated/ Actual Values and 
Mathematical Models 

f. Trend Model Selection 

g. Confidence Limits - Probable Error 

h. Forecast Line Bias 

i. Forecast of Program Maturity Factors 

j. Forecast Line Bending with Program Maturity 

k. Forecast Line Adjustment Due to Expected Changes 

l. Targeting Analysis 

Post-Trend Processing 

m. Summing of Trends 

n. Performance Calculations 

o. Calculated Deficiencies , Buyoffs, Performance Adjustments 

p. Decision Relevancy (Criticality) 

q. Trade-off Factor Derivation 

r. Cost Change Analysis 

s. Schedule Slip Analysis 

t. Reliability Change Analysis 

u. Validation of "Follow-on" Vehicles 
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A brief explanation is included to give a better understanding of what these elements are . 

a. Assessment of Mission Requirements and Trade-Off Factor Extraction - 
These data define the performance constraints and provide the necessary 
factors to facilitate relating predicted problem magnitudes to a common base. 

b. Change Analysis - The analysis of monthly changes to identify non- random 
changes which are not mathematically a part of expected growth . 

c. Normalization - Process which adjusts data for non- random changes noted 
in b, as well as for other changes which are mathematically excludable. 

d. Time Transformations - Required to adjust progress to match program 
time rather than real time. Such a transformation eliminates the problems 
of forecasting when there have been lengthy delays due to program stop- 
pages or other program time disturbances. 

e • Trend Analysis - The observed points are assessed and a curve fit made 

through these data which will be extended to the critical future date. Several 
mathematical models are available to facilitate such analyses . 

f- Trend Model Selection - The selection of the particular model which best 
fits the inherent growth patterns . 

S- Confidence Limits - Determined statistically about the mean trend line for 
both observed and predicted time periods. Probable Error is an expression 
used to describe the range of expected errors in the mean predicted weight 
at the final date and is numerically equal to the difference between the upper 
confidence limit and the mean. 

h* Forecast Line Bias - Having found the trend line, the forecast line is 
mathematically unbiased so as to extend from the most recent data point 
into the fixture time domain. 

i • Program Maturity Factors - Forecast factors made from a knowledge of 
expected progress and comparison of actual maturity with this preselected 
model. 

3 • Prediction Line Bending - Program maturity factors are in turn used to 
improve the forecast line in the future time domain to deflect the forecasts 
in accordance with known behavior. 

k. Adjust for Expected Changes - Analysis of observed monthly changes to 
permit forecast of rate of incoiporation of buyoffs which are also used to 
correct and shape the forecast line. 

1- Targeting Analysis - Final forecast results are observed and trended to 
indicate prediction accuracy. 
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m. Summary of Trends - Predictions made for pieces of the total, these are 
summed to produce predictions for the total part. 

n. Performance Calculations - Where required, performance calculations are 
performed to permit expressions of deficiencies or buyoffs on a common 
basis for direct comparison. 

o. Deficiencies, Buyoffs - The end result of Forecast Analysis is the determina- 
tion of deficiencies and buyoffs which express the required management ac- 
tion in specific numbers. 

p. Criticalities - The impact or risk associated with the noted problems are 
assigned a criticality rating by a final evaluation process which weighs 
related program constraints . 

q. Trade-Off Factor Derivation - Trade-off factors, derived during the per- 
formance calculations are provided for Management use in making adjust- 
ments between corrective actions and program constraints. 

r. Cost Change Analysis - The costs associated with making the corrective 
actions, noted as buyoffs or deficiencies, are estimated from program 
funding rates. 

s. Schedule Slippages - Estimated from current and historical relationships 
between program changes and schedule changes. 

t. Reliability Change Analysis - Reliability effects associated with required 
buyoffs can be calculated from existing Reliability Models. 

u. Validation of "Follow-On" Vehicles - The experience on Vehicle Number 1 
and the extension of such experience to the vehicles which follow. 

Numerous additional elements can be included in Forecasting Analysis for other appli- 
cations. Accordingly, any system of Forecast Analysis must be constructed with maxi- 
mum flexibility and the ability to select one or many potential analytical routines. 

The basic elements, however, are those listed above and are described in greater 
detail in subsequent chapters. 
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CHAPTER 2 

FORECAST AND APPRAISAL SYSTEM CONCEPTS 
2.1 INTRODUCTION 

While subjective evaluation almost always enters into the decision making process, 
today's complex technical programs call for an increasing amount of objective, quanti- 
fied information upon which to base decisions. The probability theories and sampling 
procedures developed by mathematicians have proven useful in quantizing decisional 
problems and are being used increasingly in large-scale industrial and Government 
programs. Evidence of this is found in the Government-wide utilization of PERT 
(Program Evaluation and Review Technique) and similar techniques. 

M the FAME approach, useful forecasts of probable events are generated by plotting 
present events and generating trend lines to determine program direction. The under- 
lying pattern and direction of these trends are then translated into quantitative fore- 
casts. This procedure is familiar enough, having wide application in instances where 
there is a continuous recurrence of similar events which lend themselves to feirly 
precise measurement. But in large programs where each unit is designed and con- 
structed by a different contractor, the discernment of an underlying trend of develop- 
ment becomes quite complicated. The following discussions illustrate how this is ac- 
complished through the FAME concept. The definition of the Forecast and Appraisal 
System is followed by a brief discussion of the techniques used to implement it. De- 
tailed description of the techniques are found in Book II . 

2.2 THE SYSTEM CONCEPT 

A basic tenet of control system theory is that the stability of a process output can be 
improved by introducing control actions proportional to the rates at which output 
errors occur. This involves anticipating the fiiture behavior of the process so that 
corrections can be made before the process goes "out of control. " Program manage- 
ment is aided by this feedback of information because forecasts of anticipated program 
behavior provide a decidedly clear advantage over a management control plan based on 
reports of current status only. A general block diagram of the Forecast and Appraisal 
System, in a program management configuration, is presented in Figure 2-1. 
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Figure 2-1. Analogous Control Loop 












The program or process to be controlled is shown providing status information to the 
system. This information is generated periodically (say, monthly) and the total of 
such data received to date is used to predict the program's status at some future time. 
These forecasts are examined in light of known constraints and interrelationships 
among the program parameters being tracked, to define and rank problems predicted 
to occur in the future. The forecasted identity of the problems and their criticality 
are based on the assumption that remedial action will be taken in the future. Manage- 
ment must then evaluate the problems in view of program requirements, constraints, 
already planned actions and various trade-off relationships to determine what new or 
modified corrective actions should be initiated to prevent the forecasted problems 
from actually occurring. Such actions are then transmitted to management at the 
program-office level for their implementation. The results will eventually be reflected 
in the status data, thereby producing revised forecasts from which the program man- 
agement can determine the effectiveness of their decisions. 

This predictive ability is thus seen to provide management with a means to prevent 
otherwise inevitable problems. The result is a smoother running program experiencing 
fewer major problems calling for quick response with associated overshoot. 

An obvious extension of such a system concept is the optimization of its performance. 
That is , development of a means for minimizing the deviations of the program status 
from a pre-established "trajectory" or to minimize the cost of program problems or 
some other function of performance. One common approach is to construct an appro- 
priate model of the process and exercise it by introducing a variety of inputs and noting 
its response. That response, optimizing the selected objective function, identifies the 
appropriate input - management directed corrective action, in this case. This capa- 
bility is not a part of FAME at this time but would be a valuable addition. 

2.2.1 GENERATING AND CONDITIONING STATUS DATA 

The feedback information generated by the program must be meaningful in the sense 
that it represents salient characteristics of the process and contains sufficient infor- 
mation for control purposes. Naturally, the variables pertinent to the prediction 
process must be defined for each program. In manufacturing and construction proc- 
esses the technical variables are readily measurable. The critical dimensions of an 
item, the number of defective items produced, etc. , are physical quantities which are 
subject to actual measuring techniques and can be controlled by the well established 
methods of statistical control. 
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There are, however, economic and technical variables which are neither easily meas- , 
ured nor amenable to established traditional control techniques. Such variables are 
especially common to research and development programs. They are characterized 
by physically unmeasurable quantities and by relatively low frequency of occurrence. 
They are subject, therefore, to the usual degree of error attached to estimated meas- 
urements. Further, they are not measured often enough to lend themselves to sam- 
pling and to statistical analysis. If, in spite of these characteristics, the observed 
variables represent significant elements in the exercise of program control, some 
means must be found to include them in the control loop. 

The fact that they can indeed be included is expressed in the following theorem which 
resulted from the application of FAME to just such data as is illustrated later in 
the book: 

Theorem - Specific measurements of program parameters such as 
weight, cost, performance, can be treated statistically 
as though extracted from a large population whose ex- 
pected value varies with time in a "natural growth" 
progression. 

Requirements are thus established not only for including these "non-physical" param- 
eters in the control loop but also for predicting their future behavior. Before con- 
sidering methods for obtaining the necessary predictions, the nature of the data or 
value of the parameters of interest deserves emphasis. 

2.2.2 DATA CHARACTERISTICS 

As a general rule relevant data will be generated at some fixed frequency. For a 
variety of reasons this frequency may vary from time to time. Also along the time 
axis (time is the usual independent variable) it is not unusual for the data to reflect 
accelerations and decelerations in the process. Thus, even at a fixed frequency, the 
data implicity include the effects of variations in the time process due to changing 
priorities , etc . For these reasons the information contained in the independent varia- 
ble must be examined carefully. 

A similar situation holds in the information content of the dependent variable, as it too 
is subject to spurious variations. This is a result of the values not being actual meas- 
urements of some physical characteristic but rather of computations based on esti- 
mates. As such, they are subject to common sources of error - incomplete informa- 
tion, arithmetical mistakes, garbled instructions , political pressures, and the like. 
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Often, these data are also affected by conscious and purposeful actions, such as 
changes in computation methods, changes in the components of a given computation, 
and changes in specifications. 

It can be accepted, then, that the data to be used as bases for prediction and control 
are two dimensional and that both dimensions are subject to the effects of both known 
and unknown (or detected) actions. A beginning strategy is to remove the effects of 
those data changes which are clearly assignable; that is, remove those changes known 
to have been caused by purposeful or non- random actions. If done in both dimensions 
data will be obtained which reflects only random actions and the basic , underly ing 
process which is to be controlled will be highlighted. The data then can be considered 
as points representing the true state of the process, modified by the effects of uniden- 
tifiable or random events. 

2.2.3 STATUS FORECASTING 

With this knowledge of data characteristics, attention can now turn to the task of fore- 
casting the future behavior of the process on the basis of the patterns assumed by the 
data. Basically, the forecasting problem is two-fold. Before any forecasting as such 
can be done the trend of the data generated to date must be discerned. That is, a model 
must be formulated which will represent the process producing the data. The model 
must relate the independent and dependent variables in a way that permits deter- 
mination of the value of the dependent variable when related to a specified value of the 
independent variable. For example, if time and project cost are the independent and 
dependent variables respectively, the model would be: cost = f(time) . Then, for any 
value of time, a corresponding cost figure could be computed. The model is formulated 
in such a way that it represents, in some optimal sense, the data in hand. The model 
is then said to represent the process. Once the model is established, predictions can 
be made about the performance of the variables. 

The usual technique for establishing a model is to "fit the curve" to the data. This 
defines the mathematical function which fits the data in some best sense . For exam- 
ple, the data might tend to represent a linear function of the variables, so a straight 
line would be "fitted, " perhaps using the method of Least Squares. Although this 
"fitting" approach is very useful and easy to implement, it lacks the power to provide 
one important piece of information — it cannot give an indication of the credibility of 
resultant prediction. To obtain such information, one must turn to a statistical treat- 
ment of the data. The statistical equivalent of "curve fitting" is Regression Analysis. 
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This te chni que, especially through the Maximum Likelihood Estimation technique, 
provides optimal model representations (i.e. , tailors a given model to the data in an 
optimal manner) plus giving confidence intervals for the prediction. 

For the purposes of this discussion, Regression Analysis will be considered to include 
aU techniques used for quantizing models on the basis of given data, including such 
techniques as Least Squares, even though it is not usually referred to as a regression 
technique . 

The general procedure followed in a regression analysis consists of assuming one or 
more forms for the model and then fitting the available data to them to quantize the 
model constants. If more than one form is considered, the quantized models can then 
be compared on the basis of some "goodness-of-fit" measure and the best one selected 
as the process model. As new data are received, the model constants may change, 
and even the "best" model form may change. Continuous review in the form of re- 
peated regression analyses is necessary as long as new data are being reported. 

A model, theoretically, can be an analytical expression. In practice, however, com- 
putational and other considerations tend to narrow the field to those containing only two 
or three constants to be evaluated. 

Typical examples are the following: 


y = 

a + 

bt 

y = 

a + 

bt 


1 


y = 

a + 

bt 

y = 

ab 1 * 



in which the constants to be evaluated are a and b. There is one formula, the poly- 
nomial, which by taking a sufficient number of terms can be used to fit any set ofdata. 
This number may be unwieldy, however, and a simpler form can be used which is 
easier to handle and yet gives no significant loss in representation. 

In some cases inspection of the plotted data can result directly in the selection of the 
best form. The data may clearly indicate a linear trend, in which case the y - a + bt 
form would be used. If the data tend to indicate a linear trend when plotted on 
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semilogarithmic paper, the form y = ae or y = aif' would be used. But, in most 
cases, the choice is not so clear and, as mentioned previously, more than one ap- 
proach must be tried and tests applied to determine the preferred one. 

Another useful regression approach for predictions is known as "stepwise" regression. 
This technique, rather than assuming a fixed or specific form for the process model, 
starts by assuming a general -n^ 1 order polynomial (say, of order six). Working down 
from the n^ 1 order, successive sets of derived coefficients are tested for significance. 
This might be done by examining the sensitivity of the fit quality to the order of the 
polynomial. In this way, an optimum order is determined below which the fit deteri- 
orates rapidly and above which fit improvement is slow. 

Recursive estimation can also be employed. This technique is generally desirable 
when the data sets are large and/ or there are numerous model constants to be quan- 
tized. If the constants (i.e. , model parameters) are denoted by 0. , let 0. , denote 
the parameters evaluated at time T. When observations are made subsequent to time 
T, say at T + 1, the recursive estimation technique computes 0. T+1 on the basis of 
0. T - This would appear to save computation time as Ihe 0. T+1 computations do not 
start "from scratch" each time but start with the stored values of the previous 
computation. 

Autoregressive techniques also hold promise. These do not start with the assumption 
of some mean value function for the model but instead utilize identifiable tendencies in 
the most recent data. That is, the later data are treated as more meaningful and so 
are used as the bases for prediction. Advances in these techniques are recent and the 
subject of considerable interest. One such advance is discussed in, "The Role of 
Spectral Analysis in Time Series Analysis" by Dr. E. Parzen. Quoting from this 
report*, "Given an observed time series of finite length, or a time series derived 
from it, one defines various sample spectral functions such as windowed sample spec- 
tral density functions and distribution functions. Their properties can be determined 
for each possible model one desires to consider for the observed time series. Con- 
sequently, they can be used to form estimates of the parameters characterizing the 
model. Further, they can be used to determine an appropriate model by comparing 


♦Technical Report No. 2, 12 July 1965. Prepared under Contract Nonr-225(80) 
(NR-042-234) for Office of Naval Research. Dr. E. Parzen - Department of Statis- 
tics, Stanford University. 
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the actual appearance of these spectral functions with their expected appearance under 
the various models; that model for which the correspondence is closest is considered • 
the most likely . 

An obvious question to ask about any computational scheme is, "How well does it 
perform?" The need to answer this is very great in the case of models designed to 
represent processes and forecast their behavior. The effectiveness of a mangement 
or control function using forecasts is determined by the quality of the forecasts. The 
only sure way of measuring this quality is to test the forecasts against the observed 
data; that is, when extrapolated beyond the observation range they are not a yardstick 
for measuring quality - this is best done within the observation range. The point can 
be visualized more easily by considering the following: 

Suppose the data set consists of 100 points. Using say the 
first 50 as a basis for forecasting how close is the fore- 
cast value of the 100th point to the observed value of the 
100tk point ? 

This example is oversimplified, of course. One must design a validation program 
that tests the particular techniques developed against the objectives. 

One might want to assess the sensitivity of the predictions to the size of the data sets, 
to the values of the model coefficients, to pre-prediction removal of non-random change 
effects, to the different models used, etc. The general term to describe this activity 
is Error Analysis, for which there are many techniques available. New ones can be 
developed to fit specific situations. 

It was mentioned earlier that one of the advantages of using maximum likelihood esti- 
mation is that it permits establishing forecast intervals. These intervals define a 
range of values about the predicted value within which the actual value is expected to 
fall a given percent of the time . The usual case is to define the 95 percent confidence 
limit. The quality of the forecasts is inversely proportional to the size of the predic- 
tion interval for a given confidence coefficient or percent level. 

To obtain a prediction interval it is necessary to know the distribution of the errors 
about the forecast line. The interval is then stated in terms of the prediction value 
and the number of error distribution standard deviations (cr ) corresponding to the de- 

V 

sired confidence coefficient. For instance, if the errors are normally distributed and 
a 95 percent confidence interval is wanted, the interval would lie between ±1.96 
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The confidence limits are then found by adding and subtracting 1.96 u to and from the 

0 

*' predicted value. If a & is not a known quantity but rather an estimated one , or if the 
sample size is small, the Student t distribution would be used. The i nt erval size is 
then a function of sample size as well, approaching the normal distribution asymp- 
totically as the sample size increases to say 30 or more. 

Greater detail on this subject is given in later chapters. It is enough at this point to 
note that this "confidence limit" measure of forecast quality is very desirable and that 
it can be obtained when Maximum Likelihood Estimation is used. 

2.2.4 PROBLEM IDENTIFICATION, RANKING, AND APPRAISAL 

When appropriate operations on the data have been performed and predicted histories 
for the process variables of interest have been generated a reasonable question to ask 
is, "How can the results be presented so as to be of most use in the management or 
control functions ?" Specific answers to this question depend upon the particular proc- 
ess in question and the management philosophy employed in its control. An analogous 
question, "What information should be presented to management?" can be approached 
by examining the generic management function and the associated decision malring 
process. In the first place, the manager needs to be continuously aware of the general 
state of the process under his control. Two of the by-products of the control loop dis- 
cussed here are : 

a. It apprises the manager of the current process state. 

b. It contributes additionally to process stability by instituting an internal 
reporting system. 

That such a control loop exists is a large plus factor in that it serves to keep manage- 
ment continuously informed. 

In addition to current information, predicted values should be presented. Even more 
important is an appropriate interpretation of the data. Problem areas must be identi- 
fied when predicted values exceed pre-specified limiting values , and criticality indi- 
cated. From this information the type and magnitude of corrective actions required 
can be determined, as well as how important and urgent it is to implement them. 

An important consideration in developing a problem criticality index is the level, with- 
in the context of the process analyzed, of the parameters being tracked. If they come 
from the first tier, (total program cost, for example) problem ranking is simple, as 
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all interrelationships are implicitly included in the parameters. But, because of this, 
identification of necessary corrective actions is very difficult. On the other hand, if • 
the par am eters of interest come from a very low tier, pinpointing problems and cor- 
rections is easy but establishing the importance of the problems is difficult in as much 
as the parametric interrelationships may be nearly all destroyed as a result of the 
level of the examination. Then there is the amount of computation involved in the pre- 
diction activity. It increases as the tier depth increases. Some compromise must be 
reached in deciding what parameters to track and at what level ? Also to be weighed 
are the relative desirability of problem identification, problem ranking and the amount 
of computation involved. 

Generally, some intermediate level is selected. After the problem areas have been 
identified, a careful assessment of the parametric interrelationships is necessary 
before any attempt at problem ranking is made. As an illustration of this principle, 
consider the following example. Suppose the problem is to track and predict the power 
requirements for a piece of electronic equipment being designed. As the design pro- 
gresses the power requirements grow and as a result it is predicted they will exceed 
pre-specified limits, thus presenting a problem. But at the same time the design is 
proceeding so that capacity is growing. It is very possible that, although a power re- 
quirement problem is currently identified, the capacity of the power supply may well 
grow at a rate which at design completion will be more than adequate. In this case, a 
real problem doesn't actually exist because of the interrelationship between power re- 
quirements and supply. 

2.3 FORECASTING TECHNIQUES 

The central role played by the prediction process is evident from the preceding des- 
cription of the Forecast and Appraisal System. Thus, it is no surprise that the sys- 
tem's effectiveness depends to a large extent upon the availability of valid means of 
forecasting. This capability is provided by mathematical techniques which formulate 
models of the process based on the set of status observations to date. These models 
describe the primary trends of the observation values and are used to generate fore- 
cast values by extrapolation. 

In the general case, one would expect data to exhibit wide variations in patterns or 
trend forms . If the system is to have the flexibility to accommodate these various 
forms in an optimal manner, either a very general modeling technique must be used 
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or a number of specific models must be made available. The concept discussed here 
is based on the latter approach. 

The following paragraphs describe the forecasting techniques employed in the applica- 
tion of FAME to the control of space vehicle design weight. The techniques are not 
limited to this application however. It also should be noted that they are not all in- 
clusive. Other particular applications, perhaps having significantly different data 
characteristics, may well use some of the techniques mentioned earlier but not em- 
ployed in this application. 

2.3.1 DATA CONDITIONING 

The first step in constructing a math model is to examine the nature of the data to be 
utilized. The parameter of interest in this case is weight growth over a measured 
period of time . Along with transmitted weight data are indicators of its relative value 
determined by that proportion of the reported weights which is Estimated, Calculated, 
or Actual . As the design program advances , the actual (measured) proportion in- 
creases while the estimated and calculated shares drop. Therefore, the data becomes 
more credible as time passes. Typical variation in the proportions of E/C/A weights 
are illustrated in Figure 2-2. The utilization and value to be derived from E/C/A 
data, whether it be in the weight, cost, or other areas, is stated here as a theorem 
and is discussed more fully in paragraph 2.3.4. 

Theorem - Forecast quality (i.e. , accuracy and validity) is 
improved through incorporation of program 
maturity factors. 

A certain amount of variation in reported weight data can be attributed to random oc- 
currences. If data exceeds an established acceptable level it is most likely due to 
design changes which are non- random in nature and which therefore must be removed 
from the data by a process-referred to as "Change Analysis. " Ordinarily, itis planned 
ahead of time to take some defined adjusting action if this acceptable level of activation 
is exceeded, knowing full well that there is some probability that the decision to adjust 
may be wrong. The decision strategy should be designed so as to minimize the proba- 
bility of making a wrong decision in either direction, that is, to adjust or not to adjust 
the data. Such considerations entail an entire field of study - decision theory - and 
cannot be elaborated upon here, hi any case, caution must be used in deciding what 
action to take regarding the effects of "non-random" events. 
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Figure 2-3 illustrates two types of effects which may result from non-random events. 
One is a wild point which falls outside the historical data envelope and the other repre- 
sents a step change in the data base. The first type can be handled by comparing the 
point's value with the historical variance. If it exceeds a specified number of standard 
deviations (the decision level) its effect will be removed or modified. Of course, if a 
reason is given for the extra large deviation, that is, its cause is assignable, there is 
reason enough for altering its influence. This is the case with the second type illus- 
trated by a discontinuity in the trend line. A fruitful area for further effort is the de- 
velopment of a means for identifying such discontinuities where the cause is not known. 



Figure 2-3 . Effects of Non-Random Events 

Removal of these effects has proven through experience with FAME application to un- 
cover the true trend of the data thus permitting more accurate forecasts. This is re- 
flected in the following theorem: 

Theorem - Forecast validity is enhanced through the removal 
of non-random effects from the observed data. 

(Application of this theorem is termed "normaliza- 
tion" in this book.) 

Those effects known to result from non-random events can be removed as follows . 
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Let the observations or data points be denoted by U x , U 2 , ... U n - When the magnitude 
of the undesired effect is determined it becomes identified as a change r. to be made 

i.L 

to the i observation U. . 

The changes are made by 

n 

\ * u t ' I r i’ k ° 1 ' 2, •••" 

i=k+l 

th 

which has the effect of raising or lowering all points prior to the 1 by the sum of this 
and all subsequent changes. If the i th point happens to be the last observation and r. 
is associated with it, all earlier points are raised or lowered by an amount equal tor.. 

Thus it is seen that not only is the true trend maintained but the predicted values will 
be based on the fact that the last point is the ’best" point. Clearly, if the (1 + l) st 
point changes in the opposite direction by the same amount, another change (r. + 1) is 
obtained which is equal to -r.. In this case, all points prior to i and i + 1 itself remain 
unchanged but the i th point is brought back into line. It should be noted that a non- 
random change is defined to be positive if the datum was forced down. Consequently, 
a positive non-random change result is subtraction of weight while a negative non- 
random change results in addition of weight. 

Now, in the case of points suspected of being non- random, that is they lie outside the 
general data envelope for no assignable reason, some test is required to establish a 
decision criterion. One way to do this is to compute the average increment U. 


and the standard deviation 


i = 2 


Each increment U. - U._ i is compared with the average U. If it deviates from the 
average by more than say ± 2 ct u it is assumed that the particular increment was not 



2-14 


completely random. Now, a non- random charge r. ' is postulated as 
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these changes are incorporated as are the others, by 


n 



i=k+l 


There are additional means which one might use to detect and incorporate non-random 
changes . These examples serve to illustrate solution techniques and provide some 
idea of what can be accomplished. 

2.3.2 MODELING METHODS 

The two commonly used techniques for adapting specific mathematical functions or 
models to sets of data are the method of least squares and the maximum likelihood 
principle. These are discussed briefly here. Detailed treatments are available in 
numerous references, for example. References 18 and 29. 

2. 3. 2.1 Least Squares 

The least squares criterion says that the best representation of a set of data is that 
which minimize the sum of the squares of the residuals. This assures small values 
for the residuals as the squared quantities are all positive. In other words, applica- 
tion of this principle yields a function which passes as closely as possible to all data 
points . 


In equation form, this criterion requires the minimization of 


n 



i=l 


where 

= observed value of the dependent variable at time t. 

A 

w. = value given by model at time t. 
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the general procedure now consists of substituting for w i the model at which the data 
are to be fitted, taking the partial derivatives of S with respect to each unknown param- 
eter in the model, equating these derivatives to zero and solving for the parameters. 

For example, if the model to be quantized is a linear form, say w. = a +bt., this pro- 
cedure yields the following values for a and b, 



This general treatment assumes implicitly that all the observed data are of equal 
worth, that is, there is no reason to believe any one point more or less than another. 
There may be cases where one or more of the points are known to be exact values. 

It is required then that the function or model produce these values and fit the remainder 
in some best sense. This results in a constrained least squares fit where the model 
is constrained to reproduce the exact values . 

Another form of least squares application is known as the weighted least squares 
wherein no points are exact but some measure of the relative worth of the data is 
available . Such weighting information may follow from program maturity . Improved 
data generating techniques, etc. With such information, the residuals can be weighted 
appropriately. 

Least squares, then, is a straightforward method which can be used to fit empirical 
data to any desired model. In some cases, however, information about the relative 
value of data is available in greater depth than is required for weighted least squares 
and since confidence intervals are also desirable, the maximum likelihood method is 
used. Maximum likelihood has the same effect as weighted least squares but is sta- 
tistically optimum and provides a basis for computing confidence intervals. 

2 . 3 . 2. 2 Maximum Likelihood 

It is often required to know the values of the parameters in a probability density func- 
tion whose general form has been assumed to represent a particular population's 
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distribution. Random samples are taken from the population in question and treated 
in such a way that estimates of the population distribution parameters are obtained. If 
this estimation process utilizes the maximum likelihood principle it can be said that 
the resulting specific distribution function has the greatest likelihood of representing 
the population's distribution. Note, however, that the general form of the function or 
model is assumed, just as in the least squares treatment. The results are analogous. 
Given the assumed form of the model, the resulting specific model (specified by the 
values assigned to the model parameters) is the fit or representation of the data. 

To illustrate this, consider the case where the random variable w is normally distrib- 
uted according to the following law 


f w ( w ^ - °) 



a V2F 


Application of the maximum likelihood technique yields estimators for the two param- 
eters n and a, 

* = i Z =i £ = jn I ^ i ' 

i=l i=l 

These estimators are expressible as functions of the model parameters . Solving the 
simultaneous equations which result from the application provides expression for the 
model parameters which, when evaluated, provide the optimal representation of the 
observed data for the model considered. 

2.3.3 SPE CIFIC MODELS 

The following paragraphs discuss four specific process models designed for a space 
vehicle weight control task. Three of them are based on assumed functional forms 
while the fourth is a modification of an autoregressive technique. This latter modifi- 
cation was required in this case due to the nature of the reported weight data. Details 
of these models can be found in Book n, Appendix C. 

2 '3.3.1 Linear Maximum Likelihood 

The method of maximum likelihood is a well established statistical principle. It is 
used here to estimate the parameters of a hypothetical population whose expected 
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value E {w .} is a linear function of time, t. , 

—l i 

Eiw.l = w. = a + bt. 

— r — i i 

as illustrated in Figure 2-4 and where a and b are unknown values to be estimated. 
It is further assumed the observed weights, w , w , . . . , w . are random (and inde 

— i — 2 — 1 

pendent) variables derived from a normally distributed population. 


Weight 

(w) 


Figure 2-4. Linear Max -Likelihood 

The observed weights, w., are assumed to be displaced randomly from the expected 
value by a displacement, e^ 

w . = w . + e. 

—l i—i 

where e. is normally distributed with zero mean and standard deviation a. . 

It is important to understand that this model does not presuppose that weight follows 
exactly along a linear growth line. Even if the "exact" weight could be accurately de- 
termined, its value would be displaced from the mean line by a random amount e^. 

The additional knowledge of the proportions of the observed weights, w . , which is 
estimated, calculated, or actually measured is introduced through the assumption 
that o\ is reduced as the proportion of w. which is estimated is decreased and the pro- 
portion of w . which is calculated or actually measured is increased. That is, the ex- 
pected excursions of e. reduces as the program matures and the estimation processes 
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give way to better calculation and actual measurements. This is represented mathe- 
' matically as 



where s is the relative standard deviation of the actual weight and nx is a weighting 
factor related to expected changes of a. with E/C/A, the Estimated, Calculated, and 
Actual proportions. 

The likelihood function, L, then becomes 



for which the maximum is found by an iterative computer solution for those values of 
the parameters a, b, and s which maximize L. 

Maximum likelihood estimates are consistent and efficient; further they are sufficient 
if sufficient statistical measurements exist. However, it is axiomatic to note that the 
results are only as good as the theory selected for the model. Other principles lead 
to different models, but in general the maximum likelihood principle represents one 
of the best general-purpose estimation tools available for Forecast Analysis. 

Probably the most limiting assumption, here, as in all four models, is that the ran- 
dom variables, w., are independent. In a number of instances the reported weights 
are identical month after month. A most likely situation if weights were independently 
reported each month. They may have been the result of a freeze of design or weight, 
or simply use of the previous months data rather than re-estimation for the current 
month. This is not attributable to randomness, but is a result of dependence. 

Over a long time period, however, the randomness of the process is more evident and 
eventually dominates the trend pattern. 

In recognition of the dependence of weight data in the short time pattern, and that the 
most likely weight at the latest reported time is the reported weight, the prediction 
line is extended from the observed weight parallel to the trend line. 
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Simply stated, the maximum likelihood methods are used to discern the so-called 
"natural" growth, and that growth line is extended from the latest data point as the 
prediction line. This process of moving from the trend line to the prediction line is 
referred to as removing the bias . 

A further advantage of maximum likelihood methods, noted earlier, is that a confidence 
interval can be readily established in the prediction range. This confidence interval 
provides the bias for determining the probable error associated with the estimated 
weight. 

2. 3. 3. 2 Non-Linear Maximum Likelihood 

Review of weight data from aerospace programs indicated that in many cases, the 
weight growth is non-linear, approaching the final weight as a horizontal asymptote. 
There are numerous reasons for such a growth pattern, such as the presence of a 
control limit as a forcing function and the application of weight control pressures as 
a program matures. To represent this situation, an exponential model was introduced 
of the form 

-ct. 

, i 

w = a - be 
i 

as illustrated in Figure 2-5 and where a, b, and c are parameters to be estimated 
and c is restricted to positive values. 



Figure 2-5. Non-Linear Max- Likelihood 
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As with Linear Maximum Likelihood, a hypothetical population is presumed as the 

source of the observed data and whose expected value E w . = y. is an exponential 

function of time, t. . As before, the observed weights w ^ , w_ 2 , w , are ass ume d 

random (and independent) variables, deviating from the trend line by a displacement e. 

— 1 

w. = w. + e . 

1 l—i 

where e . is assumed to be normally distributed with zero mean and standard devia- 
tion, O’. . 

l 

The concepts of E/ C/A are introduced as in the linear case with a weight factor, m. , 
on the relative standard deviation of the actual weights, s, through 



and the likelihood function, L, now becomes 


L = 


n 

n 

i=l 


, , w . - a + be 

1 [=1 


-ct. 

i 


m. 

i 


m. V2F 
1 


Solution of the values of a, b, c, and s which maximize L is more difficult than for the 
linear case, and the iterative computer solutions were found to converge quite slowly 
for cases of limited exponential character. 

Accordingly, a linearized approximation of the basic equation is employed through, 

w. = a - b ib. 

1 r i 

where 


-ct. 

ip. = e 1 

T i 

and is assumed to be the independent variable . The value is varied by preselect- 
ing values of c , and maximum likelihood solutions found for a and b for each value 
of c. Values of a, b, and c are then selected as those values which result in the high- 
est value of the likelihood function. 
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This exponential model, however, quite frequently converges toward an asymptote 
which is ficticious. This occurs when a set of observed data has been growing at a 
normal rate and then is followed by a period of very little change or a leveling off. 


Analysis of data during a leveling off period often results in a prediction which is far 
short of the final weight, especially if the leveling off period occurs early in the engi- 
neering development phase. Careful analysis of this type of situation is required if 
pre-mature asymptotes are to be avoided. 

The model does not allow for independent examination of the various program phases 
as related to weight growth. Program phases are similar to the seasonal changes of 
econometric analyses. Since early phases normally have a higher growth rate than 
others, a lack of sufficient observations in following phases could result in an asymp- 
tote which is considerably overestimated. 


2. 3. 3. 3 Asymptotic (Logistic) Exponential 

This Asymptotic Exponential model incorporates most of the better features of the 
linear model and the exponential model . The shaping of the logistic curve available 
through parametric variation makes this model a valuable tool. The model is repre 
sented by : 


where "a, " "b, " and "c" are parameters to be estimated and "c" is restricted to posi 
tive values so the curve will approach the value "a" asymptotically. 


A pictorial representation of the curve, Figure 2-6 is presented below. 


Weight 

(w) 
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Figure 2-6. Asymptotic (Logistic) Exponential 



This type of curve allows for little or no weight increase as a program is started, for 
an increase (or decrease) as the program moves along in time, and a 'leveling off" 
as the program completion date nears. The shape of a specific curve is influenced 
greatly by the parameters a, b, and c, which are estimated by a weighted least squares 
curve fit. 

The logistic model is quite adaptive to the data and can assume a variety of forms de- 
pending on the values of a, b, and c. If the parameter "c" is small, less than 0.1, 
the curve behaves similarly to a linear model. Eventually, the prediction curve will 
bend over to the asymptote "a." On the other han d if "c" is moderately large 
(0.2 < c < 1.0) and "b" is not large (b < 30), then the logistic curve looks very much 
like an exponential curve . 

The initial flat portion is attributed to a reasonably large "b" (> 40) and a "c" that is 
small enough so that the term "be " is not reduced to zero for small values of "t. " 

If "c" is large, then the logistic model approaches the asymptote "a" very quickly and 
proceeds horizontally to infinity. 

2. 3. 3. 4 Adaptive (Fourier) Exponential Model 

The foregoing three models are each based on the assumption that the underlying proc- 
ess can be approximated by an analytical expression of the so-called "natural" growth. 
While such models are sufficient in many cases, there are some instances where 
greater adaptibility is desired. The use of an underlying model is too restrictive to 
permit rapid sensing of the latest weight progression. Further, it is recognized that 
individual monthly reported weights are sometimes highly dependent and an autore- 
gressive type of model is needed for study of weight growth. 

The adaptive (Fourier) exponential, presupposes no underlying process, rather it 
senses the growth tendencies, with emphasis on the most recent observations and is 
therefore highly adaptive. The monthly weight increment is assumed to be the random 
variable, and the forces which cause the weight increments are assumed to be the ran- 
dom variable, and the forces which cause the weight increments are assumed to decay 
as the program matures, approaching a constant slope. Behavior of the model is 
illustrated schematically in Figure 2-7 . 
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This equation contains a constant term (w. - 6), a linear slope term and exponentially 
decaying term ce“ Q!t . The solution is found by first separating the non-linear part by 
a Fourier analysis, producing a smoothed curve through the observed data, and then 
considering the remainder of the equation as an increment process. The smoothed 
exponential curve is allowed to approach a fixed slope in the prediction range . 


By nature, the Fourier model predicts from the last point with a prediction line that 
retains some exponential character for a brief period. The confidence limits, like- 
wise emminate from the last point, since the confidence interval is selected for ex- 
pected data, and the data point is assumed to be proven with 100 percent confidence. 
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.. 2,3,4 USE OF E/C/A AS MATURITY FACTORS 

ne above discus'ion, of; prediction models indicate that the program maturity 

evi enced by E/C/A values, plays a role in quantizing the model parameters Although 

nttTat! r ^ d6ta11 1,1 C ’ “ should 

iS ““ 0017 4,16 efleCtS ° f tte E/C/A values are in- 

, manner, i.e. , the observation range only. Or, put another way the 

It n^ “° W ‘ S ^ ^ ^ ° bSerVed dat * “ d its **«»ciated E/C/A content 

akes no provision for the anticipated growth of program maturity. 

In the early phases of a program it is clear that the "value" or relative credibility of 
reported data is far from its ultimate level. But, if the small amount of such 

cIudeXrff^“lte USedt ° C ' Uanti2e ^ m0de ‘ S ' “ 18 “° re to *- 

prediction itself, as it represents a large differential. 

The use of E/C/A can be introduced into each math model in the prediction range in 
evera ways, such as changing prediction line slope with different program phases 

tiotT P d r t Cted ValUe 15 E/C/A Pr0gTesses - ° r Rusting expected standard devil- 

will haptn trE/cfinteTr 6 ' IT ”°‘ °°“ — 

values with corrections from 2 Z^E/^ M terf T "*** 

predicted E/C/A. Methods of predicting E/C/A sm 

forectt itlutT 9 ^ ^ ^ IenSt “ y ‘° * dlS ° USSed h6rc eXCept to “ ote *»* 
values t E/C/t ” “ P ”‘“ “*« * ^st predicted 

^ofEtor-:, 


w. 

1 


W i-r + P l P lff i - s,^) 


where tee superscript A denotes an adjusted prediction line and is a westing teeter, 
justed It!™ at the ad,USted PveOletion corresponding time t. by adding the ad- 

^tustett: r; for time - to a - *• 

weighting tec, or p, “si iaT ^ , “ adJUSted PredlCt “" £ ° r ^ ^ 


p i 


‘ E i + R s C t + R A, 
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here E C and A , and the percent ot E/C/A at time t and R 3 and R 4 are pr*^** 1 

« Jnta *' model is based on the underlying assumption that it the wei^i, of a • 

system has a ** percentage ot Estimated ^ 

+Vion if thp weieht is of high Actual percentage, 
rate will be greater than if the weight oi g The weight predictions , w, , 

able assumption, as investigation of Actual " Ugher per _ 1 

made for some future time t. are normally base on a a ^ passes the 

centage Estimated weight than wiU actually be present at tim , • 

percentage makeup of the functional system wei^t becomes mote 

t, mated. Since tee prediction is based on wei^its having a high per _ 

tee predictions made are for a — will not be 

centage a, time V ^ rate will not be as high as tea, predicted. 

accomplished by selecting 1, R 3> R 4 P < P for 

lated and Actual the weighting factor P t decreases . That is , Pj P^ 

i = n + l, n + 2 n+p-l,n+p. 


2 4 MODEL SELECTION 

As mentioned earlier, tee availability of an array of models Provides a 
eree of flexibility to the Forecast and Appraisal System. Having sue 

these bJ* te tee following theorem, which is the basis of tee selection 

process . 

Theorem - Forecasted values made in succeeding time 
Theorem For ^ ^ ^ on value. 

(This is refered to as "targeting in this book.) 


2.4.1 REPEATING MODE ANALYSIS 

The repeating mode technique analytes a sequential set of obscrvat.onMf.ve to sw 
points beginning with time zero) and makes forecasts for tee succeeding months^ 
teen adds tee next observed point and makes new forecasts for succee o( 

This process is repeated until all available data is eichaus e . Typical re . 

these results as a check on the attribute of consistency, i.e . , S 
suits are shown in Figure 2-8 . 
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Figure 2-8 . Repeating Mode Analysis Results 
2.4.2 TARGETING ANALYSIS 

Targeting Analysis, briefly stated, consists of a discernment of (a) whether successive 
predictions are approaching some value asymptotically and (b) if so, what the value is? 
The data needed to conduct such an analysis are generated by the repeating mode proc- 
ess and appear as illustrated in Figure 2-9 . 

# Predicted Weight 
O Observed Weight 




Ideally, the weight as a function of the number of observations should be monotonically 
increasing or decreasing toward the ultimate weight of the subsystem at the shipping 
date. However, due to errors in estimation, late design changes, and unreported 
variations in observation data, the behavior of the curve will not be consistent. In 
this case, the curve must be fitted to the points in the same manner as the prediction 
line is fitted to the observed data during the prediction phase . A nonlinear trend model 
should be used so as to enable the final "leveling off" to be simulated. The fitted curve 
then represents a "trend of the trends, " and provides an indication of the manner in 
which the trend model responds to changes in the input data. Figure 2-10 shows the 
data of Figure 2-9 together with the fitted curve . A comparison of the trended curve 
at the time of the most recent observation with the actual observed data provides a 
performance criterion that may be used in selecting the best trend mode for use with 
a given subsystem. If several trend modes produce approximately the same error, 
the actual choice of the •'best" mode may be made on the basis of other considerations 
such as consistency. 



Figure 2-10. Targeting Analysis Results 
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2 *4.3 THE MODE SET.ECTION PROCESS 

Any mode select, on scheme consists ot three steps - initialization, prediction, and 
evatnafon. The initialization phase delineates the possible trending modes, estab- 
lishes performance criteria and associated tolerances, and defines the prediction and 
evaluation procedures which follow. The prediction phase encompasses the actual 
trending computations together with such intermediate computations as may be re- 
quired. Finally, in the evaluation phase, the results of the various prediction compu- 
tations are analyzed within the framework established by the initialization phase. 

2,4. 3.1 Initialization 

The initialization phase, illustrated in Figure 2-11 establishes the order in which the 
prediction schemes are to be applied. The sequence of operations is as follows. 

A priority list is established which contains the names of all trend modes which have 
been made available for use. (Normally this will amount to four - the Unear, non- 
linear, Fourier, and logistic models.) The relative priority of each mode can be spe- 
cified by the user if desired. Should two modes prove to be acceptable, the one with 
the higher priority is always chosen. If only one mode is specified, the mode selection 
process is bypassed. The last month's mode is always placed at the top of the priority 
list, provided that mode has been included as an allowable method. If the one month 
prediction agrees with the observed data for that interval, within some prespecified 
tolerance, no further computations or predictions are made and last month's mode is 
retamed. If fids mode does not produce acceptable results, other modes wiU be tried 
m an effort to find an acceptable mode. Control is then passed to a routing routine 
which executes the proper trending programs. 

2. 4. 3. 2 Prediction 

The prediction phase is diagramed in Figure 2-12. This phase includes the actual 
trend prediction process. The first trend estabfished predicts the system weight to 
ship date. If the particular trend mode selected happens to be last month's mode and 
the one month extrapolation agrees with the observed value, control is immediately 
transferred to the decision module . 

For all other cases, the next test is to compare the predicted ship weight with a simi- 
lar computation from last month. If the difference is within tolerance, the prediction 
is run in the repeating mode form to generate data for later evaluation. 
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Figure 2-11. Initialization of Trend Mode Selection 












Figure 2-12 . Prediction Phase of Trend Mode Selection 
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The prediction interval for this part of the procedure is one month. If the ship weights ^ 
do not compare within tolerance, the mode is summarily rejected from further 
consideration. 

2. 4. 3. 3 Evaluation 

The evaluation of the prediction data is performed in the decision module shown in 
Figure 2-13. The decision module makes the final selection of the ,r best trending 
mode. If the mode is specified manually, that mode is, by definition, "best." Like- 
wise, if no mode satisfies the tolerance specifications, the decision is also trivial. 

In this case a straight line is assumed through this month's observed data and last 
month's predicted ship weight. An appropriate comment identifies this condition to 
the user. 

In the non-trivial case in which several prediction modes need to be compared, the 
procedure is more complicated. As a result of the repeating mode computations 
described in the previous selection, several values of predicted weight (as a function 
of number of observed data points) are available for each trend mode still under con- 
sideration. This data, in turn, forms the input to a nonlinear trend program which 
computes the predicted weight. The predicted value can be regarded as a "trend of 
the trends." The predicted weight is compared to the Actual (observed) weight, and 
the first mode on the priority list that produces a result within tolerance is selected 
as the "best mode. All the intermediate data plus the final results are available for 
further analysis if desired. 

2.5 SUMMARY 

The foregoing discussions have described the Forecast and Analysis System and its 
elements. It was seen that the predictive or forecasting ability is the heart of the 
system and is that element having the greatest stabilizing effect on program perform- 
ance. The salient characteristics of the data describing the program status were seen 
to exert a large influence on the approach taken to implementing the system - in par- 
ticular, the data conditioning and modeling techniques employed. 

These concepts were made more meaningful by illustrating their application to a spe- 
cific program management problem. The next chapter expands upon this application. 
Attention is turned to the auxiliary techniques and computations which, together with 
the forecasting ability, form the whole of the Forecasting and Appraisal System. 
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CHAPTER 3 


APPLICATION OF FAME FOR WEIGHT/PERFORMANCE CONTROL 
3.1 INTRODUCTION 

The techniques of FAME are best understood by describing their use in a specific ap- 
plication. The first application of FAME was for weight/performance control on the 
Apollo Program where a high premium was placed on economy of weight and volume 
in the design of space vehicles. Therefore each spacecraft, launch vehicle, and their 
respective functional systems was vigorously evaluated to assure that weight/ 
performance values were within established control limits. 

Unplanned weight growth results either in degradation of performance goals or ex- 
ceeding the capability of the launch vehicle with its attendant delays. Typical was the 
weight growth problem present throughout the Mercury Program, illustrated in 
Figure 3-1. 



CALENDAR YIAR 


Figure 3-1. Mercury Spacecraft Weight Chronology 
from Mercury Summary Report 


The lesson here is that proper planning must account for the inevitable weight growth 
in the design of high performance spacecraft , since the consequences of not pl anning 
for it are either a degradation of the performance goals or exceeding the capability 
of the launch vehicle with its attendant delays. 
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Similar weight growth tendencies were evident in the Apollo Program space vehicles. 

The Apollo Program Weight/Performance Management System, by using FAME, pro- 
vided a systematic procedure which assured the detection of potential weight growth 
tendencies. Continuous review and assessment of Apollo weight and performance was 
accomplished through the use of high-speed digital computers illustrated in Figure 3-2. 


CONTINUOUS REVIEW 


Figure 3-2. Continuous Review and Assessment Through 
Use of Digital Computers 

Weight/Performance management was and is an integral part of the Apollo Program 
Office (Figure 3-3), and provided a "continuous loop" of data and evaluation flow. 

The total Weight/Performance Management System is founded on four basic principles. 

• Requirements 

• Data Flow 

• Assessments 

• Management Actions 

The establishment of weight control requirements in the Apollo Program Specification 
and the Flight Mission Assignments document (Level 1) provides a base against which 
weight/performance progress can be measured. 

Level 1 program requirements are further detailed into Level 2 technical requirements by 
the Centers and are implemented at Level 3 by the Contractors as illustrated in Figure 3-4 . 
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Figure 3-3. Apollo Weight/ Performance Management in Relation 
to Program Organization 
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Figure 3-4. Program Requirements Versus Level 

The Weight/Performance Management System provides for a data flow system which 
assures the timely transmittal of pertinent information between Contractors and Cen- 
ters, Centers and the Apollo Program Office and the Apollo Program Office and 
other headquarter elements (Figure 3-5) . This is accomplished through the 
utilization of those data flow requirements as detailed in the Apollo Program Mass 

Properties Standard and the Weight/Performance Data Submittal Requirements 
document. 
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APOLLO PROGRAM OFFICE 



Figure 3-5. Data Flow System 

Such acquired data is utilized for weight/performance assessments which define the 
current and anticipated program status. 


The contractor status reports received by the Centers are evaluated against Level 2 
requirements. These results are forwarded to the Apollo Program Office for evalua- 
tion against Level 1 requirements (Figure 3-6). 


AP0L10 PROGRAM OFFICE CENTERS 


CONTRACTORS 



Figure 3-6. Status Reporting and Assessments 
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.. The results of these assessments are documented in the Apollo Space Vehicle Forecast 
Analysis and Management Evaluation Report (Figure 3-7). 



Figure 3-7. FAME Reports Complete the Data Flow System 


The Weight/Performance Forecast Analysis and Management Evaluation Report is dis- 
tributed to key ma n agers in the Apollo Program Office in Washington, D. C. , and ap- 
plicable NASA Centers. The Weight/Performance Management System is in itself an 
action essential to the achievement of all weight/performance goals. 

3.2 REQUIREMENTS 

The Apollo Program Specification, a Level 1 document, delineates performance, de- 
sign, and test requirements for the Apollo Program. The body of the specification 
applies to the Apollo Program equipment to be provided for operational versions of 
space vehicles leading to and including the lunar landing missions. 


Appendices to the body of the specification delineate the performance, design, and test 
requirements as they apply to Apollo Program equipment to be used on individual mis- 
sions as specified in the Flight Mission Assignments document. 


The Control Weights Requirements document establishes the control weights for launch 
vehicles and spacecraft for each space vehicle mission presented in the Flight Mission 
Assignments document. 








The establishment of these control weight requirements and mission requirements pro-.- 
vide a base against which weight/performance progress can be measured. The control 
weight is defined as the minimum acceptable value, when evaluating launch vehicle pay- 
load capability, and as the maximum acceptable value when evaluating spacecraft and 
individual launch vehicle stage weights. 

3.3 DATA FLOW AND PROCESSING 

An essential aspect of the Forecast and Appraisal System is the flow of data between 
the various management, contracting, and contractor agencies involved in the Apollo 
Program. For weight and performance data, information is transmitted down to the 
level of spacecraft and launch vehicle functional systems. Apollo Program managers 
must be able to control weight at the functional system level to provide effective con- 
trol. The status reports available to major program management elements must be 
timely and consistent. 

3.3.1 DATA SUBMITTAL REQUIREMENTS 

In the Apollo Program, a Weight/Performance Management System provides for trans- 
mittal of information between Contractors and the NASA Centers, between the NASA 
Centers and the Apollo Program Office, and between the Apollo Program Office and 
other NASA headquarters elements. This is graphically illustrated in Figure 3-8. 

Data submittal requirements are documented in the Apollo Program Mass Properties 
Standard and the Apollo Program Weight/Performance Data Submittal Requirements 
document. The Mass Properties Standard requires that Contractors report current 
weight, performance, center of gravity, and mass moment of inertia data to the Cen- 
ters. The Weight/Performance Data Submittal Requirements document requires that 
the Centers report to the Apollo Program Office current stage and module weight, pro- 
pellant loading, and engine performance data, as well as launch vehicle payload capa- 
bility and total spacecraft weight. A detailed description of these documents is included 
in the reference section of Book I. 

3.3.2 DESCRIPTION OF INPUT DATA 

Various kinds of information are included in the data submittals and many of the char- 
acteristics of the data are known. Table 3-1 lists the known facts about examined data. 
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Figure 3-8. Data Submittal Requirements Flow 
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Table 3-1 

Known Facts About Examined Data 


2 . 


1 . Data is formally reported once a 
month. 

Reported data is a result oi 

(a) Actual weight measurement. 

(b) Calculations based on detail. 

(c) Estimations based on design lay- 
outs, i.e., also calculated but 
based on less information than 
that found on detail drawings. 

3. Reported data is accomplished by 
change analyses. 

4. Authorized, Pending, Planned, and 
Proposed Weight/Performance 
change information is submitted 
monthly. 

5 . Data is reported on a functional 
system basis . 

6. There are schedules for hardware 
development (design, manufacture, 
test, checkout, etc.) 

7. There is interdependence between 
functional systems. 

8. There is interdependence between 
stages and modules. 

Functional system development 
schedules are different. 

Functional system design criteria 
are defined in specifications and 
contractual documentation. 

Design reviews are held quarterly 
(approximate) with resultant design 
changes reflected in change data. 

12. Actual weight data has relatively 
small error. 

Calculated weight data has modest 
error. 

Estimated weight data has high 
error. 

13. Data of early phases subject to high 
random variation (due to refinements 
in design criteria which were previ- 
ously approximated; first and second 
level optimization; trade-offs be- 
tween systems , and previously ig- 
nored secondary design conditions 
becoming primary design conditions) . 


9. 


11 . 


18. 


19. 


23 


24 


Weight accounting is a daily proce- 
dure and if a daily procedure audit 
were to be made and the results 
plotted, a waveform pattern would 
be evident as opposed to month saw- 
tooth trend. , , . . , 

The effectivity (i.e. , schedule) ot 
authorized, pending, and planned 
changes can be established, thus 
providing for knowledge of future 
happenings. 

Weight data is dependent on engi 
neering releases. Releases are 
planned and scheduled. Weight is, 
therefore, time dependent. 

Weight data is supplemented by 
% Actual, % Calculated, and % Es- 
timated information. 

Government furnished equipment is 
included in weights and is not nor- 
mally subjected to strict weight 
control requirements. 

Contractors are contractually obii 
gated to specification weights. 

Design constraints exist (e.g. , tank 
capacities, size restrictions, fac- 
tors of safety, etc.). , 

Month-to-month reporting frequent- 
ly reflects step function when plot- 
ted (can be attributed to stretchout 
of schedule, several months of 
status quo due to major redesign 
effort, design nearing completion, 
or changes which are sporadic and 

far apart). , . ... 

The number and magnitude of weignt 
changes decrease rapidly after the 
design and manufacturing phases. 
Major changes can occur as a re- 
sult of testing effort waveform pat- 
tern begins to resemble aharmomc. 
Reported status of Estimated, Cal- 
culated, and Actual data does not 
necessarily coincide with reported 
engineering releases. 
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Some of the features of the data can be summarized as follows: 

1. Weights derived from layouts and sketches are referred to as Estimated 
weights. Weights calculated from officially released detail drawings are 
referred to as Calculated weights. Weights determined by measurements, 
with inherent instrument and part tolerance errors of the actual hardware 
are referred to as Actual weights. The class of weight is reported each 
month for each functional system. Each class has its own inherent error, 
but nevertheless gives to the reported weights a built-in statistical weighing 
factor which reflects program maturity. 

2. The behavior of certain functional systems can be traced to system inter- 
dependence. For example, electrical power system weight is a direct func- 
tion of power supply and power demand. Another example is structure, 
where weight changes or changes in design criteria in other systems are 
frequently reflected in structural load changes , and hence structural weight 
changes. 

3. The relationship of the reported data to the program schedule phase (i.e. , 
design, engineering, manufacturing, test, refurnishment, checkout, deliv- 
ery, etc.) provides a measure of program maturity. A correlation can be 
made between the reported data and the current schedule phase. 

4. The change analysis data tells why a weight change has occurred. This data 
provides the basis for the normalization of previously reported weight data. 
Normalization here is analagous to the removal of seasonal effects quite 
frequently found in econometric data. Normalization contributes to the de- 
termination of a true rate of weight growth by eliminating effects caused by 
transfer of weights between functional systems. 

5 . Authorized changes are those which have completed the engineering approval 
cycle but have not been officially incorporated via an engineering release. 
The approximate dates when these changes will become effective can be es- 
tablished (± one month). Pending changes are those which are in the approv- 
al cycle. Approximate effective dates can also be established for pending 
changes (±two months). Planned changes are those which are still being re- 
viewed before going through the approval cycle. Approximations of the ef- 
fective dates of this type change are more difficult. Proposed changes must 
be studied to determine feasibility, impact, and actual worth in terms of 
weight and performance. The chances of survival for proposed changes are 
small unless they meet predetermined standards. This type of change is 
usually held in reserve until circumstances warrant its incorporation. 
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3.3.3 PRE-PROCESSING OF DATA 


3. 3. 3.1 Weight Data 

Current weights , Estimated, Calculated, and Actual weight percentages, plus applica- 
ble non- random weight changes, are extracted for the launch vehicles and spacecrafts 
from their respective documents. 

a. Extracted launch vehicle weight information specifically includes: 

(1) Payload capabilities, stage and functional system weights for the 
Saturn IB and Saturn V missions. 

(2) Estimated, Calculated, and Actual weight percentages at the stage 
and functional system level . 

(3) Non-random weight changes at the functional system level. 

b. Extracted spacecraft weight information specifically includes: 

(1) Earth orbit injection weights, translunar injection weights, module, 
and functional system weights for the Block I and Block n spacecraft. 

(2) Estimated, Calculated, and Actual weight percentages at the space- 
craft module and functional system level. 

(3) Non-random weight changes at the functional system level. 

3. 3. 3. 2 Schedule Data 

Launch vehicle and spacecraft schedule information is extracted on a mission-by- 
mission basis. 

a. Extracted launch vehicle schedule information specifically includes: 

(1) The long lead-time procurement date (date when action must be 
started to procure long lead-time items in order to assure sched- 
uled mission completion). 

(2) Start of fabrication and assembly. 

(3) Start inplant checkout. 

(4) Start acceptance tests. 

(5) Start of refumishment and checkout. 

(6) Ship date to KSC. 

(7) Launch data for indicated launch vehicle. 

b. Extracted spacecraft schedule information specifically includes: 

(1) Start of subsystems. 

(2) Start of subsystem installation. 

(3) Start of inplant checkout. 


3-10 



(4) Ship date to KSC. 

(5) Launch data for indicated spacecraft. 


It should be pointed out that a single shipping date is generally acceptable for all func- 
tional systems and stages or modules of a given launch vehicle or spacecraft. This 
date will be the latest shipping date of any of the components of that particular vehicle 
or spacecraft. 

3. 3. 3. 3 Mission Data 

Mission data consists of factors that are influenced by the over— all mi ssion require— 
ments (i.e., mission trajectory, mission plan, mission goals , etc.). 

a. Control Weights - A control weight is a limit which if exceeded may 
cause mission failure or serious degradation. Control weights are es- 
tablished by the apportioning of launch vehicle lifting capability and total 
spacecraft weight. Control weights for launch vehicle payload capa- 
bilities, spacecraft injection weights, and stage and module gross 
weights are currently being used in the Forecast Analysis Program. 

b. Trade-off Factors - The trade-off factor is used to express the stage 
and module growth in terms of payload capability. These factors are 
determined for each stage and module of a mission by using control 
weights with compatible mission data. 

3. 3. 3.4 Levels of Comparison 

There are several levels of system breakdown for an Apollo space vehicle at which the 
vehicle can be studied or compared with other vehicles. These breakdown levels in- 
clude, in ascending order, the functional system level, the stage or module level, the 
launch vehicle or spacecraft level, and the total space vehicle level. A typical func- 
tional system breakdown is shown in Table 3-2. The lowest level at which management 
must maintain its awareness of status and problems determines the degree of visibility 
which must be provided by Forecast Analysis. 

For the Apollo Program this level is the functional system level. There are several 
reasons for this. First, the growth rates for similar functional systems differ from 
one stage or module to another. For example, the service module structure may grow 
faster (or slower) than the command module structure. In cases where mission re- 
quirements (and where sufficient data is available) , it has been found that functional 
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Table 3-2 

Typical Functional System Breakdown 


System 

Components 

Structure 

Stages, Interstages, Crew Compartments, etc. 

Landing and Docking 

Tending Gear, Docking Structure, Flotation 
Systems 

Protection Systems 

Ablator, Acoustic, Meteorite, Radiation 

Personnel Accommodations 

Furnishings, Seats, Food, etc. 

Propulsion 

Engines, Plumbing, Pressurization 

Environmental Control 

Temperature, Pressure, Fire 

Guidance and Navigation 

Inertial, Stellar, Planetary 

Electrical Power 

Fuel Cells, Batteries, Wiring, etc. 

Instrumentation 

Sensors, Antenna, Transmitters, etc. 

Communications 

Tranceivers, Antenna, Cameras, etc. 

Personnel 

Crew, Suits, Life Support Equipment, etc. 

Cargo 

Scientific Instruments, Experiments 

Propellant Reserves 

Flight Performance, Launch Window Propellant 
Utilization, etc. 

Residual Propellants 

Pressurants, Trapped Propellants, Bias, etc. 

Propellants 

Thrust Buildup and Decay, and Full Thrust 
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system growth rates for a given functional system differ between two manufactured 
models of the same stage or module. In other words, structures in the first service 
module to be manufactured might have had a higher (or lower) growth rate than struc- 
tures in the second service module. 

Another reason for studying Apollo vehicles at the functional system level is that change 
analysis data is reported at this level. As already noted, change analysis data is used 
to normalize previous weight data. Normalization is essential to the Forecast Analysis 
process because it removes the effect of non- random changes, which then allows the 
determination of meaningful weight growth rates. Without normalization, the results 
of Forecast Analysis would be misleading, and without change analysis at the func- 
tional system level, there could be no normalization. 

A third reason for functional system studies is that problem areas are reported at the 
functional system level by the Contractors and the Centers. Systems or equipment 
which have excessive weight growth rates or which are already overweight are usually 
discussed at the functional system level. 

For these several reasons the lowest level at which Forecast Analysis techniques are 
applied, and for which results are presented to management, is the functional system 
level. 


3.4 NORMALIZATION OF INPUT DATA 

To arrive at valid forecast of future occurrences by extrapolation from data, it is nec- 
essary to assure that the data is of a consistent nature. 

It is not practical , for example , to base a forecast on historical status data which in- 
volves a system that has been changed from one with a single functional purpose to one 
having a multiple functional purpose. The multiple purpose system will, in all proba- 
bility, have more equipment and support structure, different thermal requirements, 
testing procedures, etc. , all of which affect system weight, cost, schedule, and 
reliability. 

Before the status data on the system can be used as a base for a forecast, an analysis 
must be made of the changes that have taken place within the system throughout its 
history. The purpose of the analysis is to normalize the reported information by mak- 
ing adjustments to remove the effect of non-random change. In this way, a forecast of 
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future status will be based upon only random changes resulting from the normal design 
cycle. The data will then more accurately reflect the actual status of the multiple pur- 
pose system mentioned in our example - treating it as though it had always been such 
a system. 

In performing Forecast Analysis of weight/performance growth, many reports and 
specifications are reviewed. Weight data is extracted and entered in the computer. 
Concurrently, weight changes since the last status are assessed to isolate those 
changes considered "non- random" and to eliminate the perturbation effects of those 
changes. This procedure normalizes weight/performance data. 

The normalization process is necessary to determine true weight growth and has a 
marked influence on prediction line characteristics, as demonstrated in Figure 3-9. 
Shown are two forecast lines of interest, indicated by B and C. Forecast line B rep- 
resents the logical extension of a trend line through reported, un-normalized data. 

The non-random change A has not been eliminated and so forecasts are forced away 
from the normal growth expectations. Line C represents the forecast line after re- 
moval of effects of non-random change A. This is accomplished by adjusting each 
previously reported data point by an amount equal to the weight increment at A. The 
adjustment is indicated by the black dots. Line C now represents a normalized trend 
line based upon consistent historical weight information which affords confidence in 
the forecast of expected natural weight growth of the system. The wide divergence of 
forecast lines B and C indicate substantially different weights at shipping date, (indi- 
cated by the arrow) , and illustrate the importance of data normalization. 

To achieve valid forecasts representing expected natural weight growth patterns of the 
many Apollo functional systems, sound rationale must be consistently employed when 
performing normalization procedures. The logic and ground rules of normalization 
steps are illustrated in Figure 3-10. 

The type of data extracted is shown to the left of Figure 3-10. As indicated, the nor- 
malization process is concerned with stage and module functional system change data. 
A determination that any change is random excludes the possibility of non- randomness 
and stops the process for that change, because the effects of random changes only are 
included in the Forecast Analysis. Changes arising from reallocation of weight be- 
tween functional systems are readily discerned as non- random and so are eliminated 
by Ground Rule 1 shown in Figure 3-10. The next four questions in the figure are 
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Figure 3-9. Normalization Process 


concerned with changes which represent basic inequalities in over-all criteria. Non- 
valid values of historical weight are discerned through applying these questions and 
can be removed as non-random in accordance with Ground Rule 2. The sixth and final 
question leads to removal of those changes which are a result of a major buy-off to 
meet specification requirements, as established by Ground Rule 3. Identified non- 
random changes then are stored in the computer weight data file, along with the other 
weight data and, as forecast runs are called for, adjustments are automatically made 
for these changes, resulting in - for each functional system - consistent historical 
data which can now serve as a basis for valid forecasts. 

3.5 FORECAST MODEL SELECTION 

3.5.1 GENERAL 

A critical operation in Forecast Analysis is the selection of the appropriate trend model 
to match data behavior. The rational for model selection is described in a general 
sense in Chapter 2 and will be discussed here as applied to weight/performance analysis. 

There are four models as illustrated in Figure 3-11, each with different characteris- 
tics to match weight growths of different intrinsic nature. In addition, a line of con- 
stant slope maybe used when indicated by experience with a particular type of component. 
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Figure 3-10. Data Normalization Logic, Assessment of Change Data 


















Time — ► 

Figure 3-11. Trend Model Characteristics 
3.5.2 SE LE CTION OF TREND MODEL 

The selection of a Trend Model can be performed by either manual or automatic means , 
this choice usually being determined by the amount of data available. In the early stages 
of weight growth it is very difficult to determine which trend mode the growth will as- 
sume. Experience has indicated that at least six weight observations are required be- 
fore trending can be attempted; and twelve or more observations are required before 
a trend of the trends is meaningful. 


As an example, assume that at a given time, six weight observations have been made 
to date. On one of these six observations an increase in weight is indicated. What 
prediction trend model should be selected ? Past experience and sound engineering 
judgment are helpful factors at this point. The linear or non-linear trend models are 
generally the most logical selection since only six weight observations are available , 
the final selection depending on engineering judgment and experience. The use of a 
constant slope model is avoided if possible or replaced as soon as sufficient data be- 
comes available in favor of either the Logistic or Non-linear Model, unless from past 
experience it has been found that this particular item responds to one of the other trend 
modes. However, the fact remains that it is difficult to objectively select trend models 
until approximately twelve observations are available. 
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Figure 3-12 summarizes in capsule form the logic used for model selection, particu- 
larly for the selection where minimal data is available in the early stages of a program .. 
Blocks 1 and 3 in Figure 3-12 apply where there is sufficient prior knowledge for selec- 
tion, thereby by-passing the remainder of the selection considerations. Assuming such 
is not the case, Block 5 is entered and last months trend mode (if any) is selected and 
checked to assure ship weight is within prescribed bounds. If there is an unexpectedly 
large change, or if trend mode was not selected the previous month, then Block 6 is 

entered and analysis is based on a repeating mode analysis for Blocks 9 and 10. 

3.5.3 TREND OF THE TRENDS 

When the number of weight observations increase to at least twelve, targeting analysis 
using a trend of the trends can be made to determine which trend mode is the most 
representative or appropriate. Figure 3-13 illustrates targeting analysis for weight/ 
performance analysis. Assume at time "L," after twelve observations have been 
made, that the item's weight is known to be 10,000 pounds. A comparison now can be 
made between the linear and the non-linear trend modes to determine which of the two 
has made the best prediction as to what the predicted weight of the item will be at 
time "L. " From Figure 3-13 it can be seen what each of the predicted weights were 
for time "L" when made at times "A," "B," "C," etc. , and the final predictions at 
time "L. ” Say the non-linear trend of the Linear Trend Model shows a 10,400 pound 
prediction and the non-linear trend of the Non-linear Trend Model shows a prediction 
of 9,900 pounds at time "L." From this trend of trends it can be concluded that at 
this early stage a selection of the Non-linear Trend Model will provide the best results 
for forecasting the continuing weight growth of the item as it moves toward the ship 
ment date, and therefore the Non-linear Trend Model would be selected in this example. 

3.6 FORECAST ANALYSIS 
3.6.1 ANALYSIS 

Comparative results of the four forecasting models are illustrated for a representative 
functional system on Figures 3-14 and 3-15- Figure 3-14 illustrates graphically the 
resulting curves fit through the normalized data. The heavy lines connect the actual 
reported data points. The distance between the actual points and the trend line shown 
by a heavy dashed line is due to normalization of the data. The forecast line, shown 
as a light dashed line , is extended from the last data point to remove the bias from the 
trend line. The purpose of this unbiasing in this manner is to improve the models with 
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Figure 3-12. Application of Trend Mode Selection Logic for Weight/Performance Analysis 
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Fisure 3-13. Illustration of Targeting Analysis for Weight/Performance Analysis 
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respect to time dependency and the recognition of the fact that the most recent data 
point is the most valuable information at any time. 

The comparative data resulting from calculation of each of the prediction models is 
tabulated in Figure 3-15. 


Interpretation of these charts is assisted by a discussion of content of each col umn . 
Referring to Figure 3-16, these columns are numbered from one to ten, representing 
the flow of calculations from initial weight data to forecast results as follows: 


Column 1. 
Column 2. 
Column 3. 
Column 4. 
Column 5a. 
Column 5b. 
Column 6. 
Column 7. 

Column 8. 
Column 9. 


Column 10. 


Calculations start with the data as a chronological tabulation of all 
weights and E/C/A weight percentages in the observed range. 

The observed non- random changes, as deduced from the reported 
data, are listed at the month of introduction. 

Other non-random changes, if any, are data points found to exceed 
a reasonable predetermined limit. 

The normalized weights, Column4, are then the resultant of Col- 
umns 1 to 3. 

The mean trend line is a tabulation of results produced by tr ending 
of normalized data by a particular computer model. 

The mean trend line is extrapolated to the ship date of the particu- 
lar system being trended. 

The upper 95 percent confidence limit is calculated about the mean 
trend line, using the parameters of the particular computer model. 
Normalized weight values in the forecast range are values having 
the same slope as the mean trend line in that range. However, 
they are taken from the last observed weight. (See Figure 3-15.) 
Other non-random changes in the forecast range are generated for 
the forecast range, using predetermined criteria. 

The forecast values are the values of the normalized line in the 
forecast range , and non-random changes in the forecast range are 
incorporated into them. The Estimated, Calculated, and Actual 
weight percentages in the forecast range are projected interpolation 
from current values to values at the ship date. 

The average weight growth is the difference between the final fore- 
cast weight value and the current weight value with this difference 
divided by the time span in months. 
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3.6.2 FORECAST MODEL COMPARISON 

Comparison between the various charts and graphs of Figures 3-14 and 3-15 can now 
be made to illustrate differences between the four models. The linear results, which 
are nor m a ll y similar to a linear least squares tend to produce excessive weights at 
ship date. For this particular set of data, the other models produced results which 
ranged in descending order from the 7005 pounds of the Fourier to the 6379 pounds 
forecast by the non-linear to the 6273 pounds forecast by the logistic model. The 
characteristics of these models can be observed here from the nature of the trend data 
and forecast lines. The model which best represents the expected growth at this time 
was identified after study and evaluation, as described in paragraph 3.5, was the 
Fourier Model. Non-stationarity of weight growth behavior, however, necessitates 
constant surveillance and at some future date one of the other models might be pre- 
ferred due to changing behavior patterns. 

3.6.3 FORECAST LINE ADJUSTMENT DUE TO EXPECTED FUTURE CHANGES 

Through the normal design, manufacture, and test cycle weight changes occur. The 
nature of these changes may be the result of improved design, material substitutions, 
removal or addition of parts, and many others. There are always proposed and pend- 
ing changes to be evaluated for authorization, and possible incorporation into the item. 

Use of the history of weight changes aids in the forecast of future weight growth and 
required buy-offs, based on proposed changes, by the development of correction fac- 
tors. These correction factors are applied to the proposed weight changes on the item 
and provide management with a higher confidence level when analyzing and forecasting 
future weight growth. 

For example, assume that weight changes amounting to a decrease of one thou san d 
pounds has been proposed on an item in the process of being manufactured. It is im- 
portant for management to know the probability of these changes being made , if made , 
the amount of weight changes when incorporated, and how long it takes to process and 
incorporate the changes. From an analysis of the history of weight changes these 
correction factors are obtained for appraisal in the following maimer. Assume that 
on a particular item, the records show that seventy-five proposed changes have been 
made and sixty have been authorized and incorporated in the item. Of the sixty incor- 
porated changes , it has been found that the average time from proposed to incorporated 
change has been two months. Further study also indicates that the average percentage 
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of the original value of the proposed weight change when incorporated has been eighty 
five percent of the original estimate. 


From this information, it can now be concluded that: 

a. There is an 80 percent probability that the proposed changes will be 

incorporated. 

b. The proposed changes that are authorized will take two months to be 
incorporated. 

c. Instead of the 1000 pounds that was proposed, the forecasted weight when 

incorporated, two months hence, will be 1000 x .80 x .85 680 pounds. 

This example is illustrated in Figure 3-17 shown below. 



Figure 3-17. Forecast Line Adjustment Using Correction Factors 

Although, the example offered here is for an item with a decremental decrease in 
weight, similar correction factors could be applied as the rate of expected changes. 
In this case the slope of the forecast curve would be adjusted to include incorporation 
of forecasted buy-offs based on appraisal of prior rate of buy-off. While specific 
forecast rates of buy-off will depend on the specific program being considered, there 
are several observations of general interest. At first thought it would seem that an 
ideal progr am would be one that would run from design concept to shipment without 
c hang es. However, changes' will and do occur for various reasons related generally 
to the achievement of an efficient design with minimum cost. While equipment is in 
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the process of being designed, changes are relatively easy to make and can be made at 
mi n i m um cost. As the equipment progresses into the fabrication and test stages, 
changes become progressively more difficult and expensive. Also, extensive changes 
late in a program may lead to schedule slippage and possible delay in mission. 

Figure 3-18 shows change curves for two items "A” and "B," as they progress through 
design, fabrication and test to shipment. Curve "A" is a more desirous change pattern 
for a program to follow than that in curve "B. " Curve "A" shows that the majority of 
changes are made on the item while still in the design and early fabrication stages and 
then a gradual decrease in changes, or possible buy-offs, through fabrication and 
test. As shipment date approaches, curve "A" would indicate that only minor changes, 
as a result of test, are required. Such a curve would indicate close scrutiny on the 
part of management to assure program success, with changes and buy-offs being made 
at minimum cost and without endangering the scheduled shipment of the items. 


Curve "B, " on the other hand, indicates a poorly managed, or possibly a "crash" type 
program. Fewer changes are made during the design stage, with the majority of 
changes occurring during fabrication and test. A program such as this, indicates a 
"wait and see" type approach. The close scrutiny that was observed in the program 
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represented by' curve "A" does not exist here. Potential changes and buy-offs are al- 
lowed to build up through the fabrication stages and into test. When changes must be 
made, and shipment is close at hand, it is possible that changes will have to be made 
without proper planning and evaluation, at excessive cost, and with the possibility that 
schedule slippage will result. 

3.7 BUILDUP AND PERFORMANCE CALCULATIONS 

3.7.1 BUILDUP OF FORECAST ANALYSIS 

3. 7. 1.1 Identification of Application 

The Apollo missions will be performed with a launch vehicle comprised of from four 
to six major stages and a spacecraft with from four to six modules. Each stage or 
module will have from three to sixteen functional systems. These systems, in turn, 
will have different performance requirements , different intrinsic weight growths and 
performance degradations . A breakdown of the units in each flight is shown in 
Tables 3-3, 3-4, and 3-5. For reasons explained in this chapter, the functional sys- 
tems are trended rather than the payload weight and launch vehicle capability. Be- 
cause of this, the predicted launch capability and spacecraft weight must be "built up ' 
from the predicted weight of the functional systems. This procedure requires detailed 
knowledge of the particular missions and their vehicles so that appropriate mathemati- 
cal models may be constructed. 

An example of how mission details influence the effect of functional- system weight change s 
on total spacecraft weight, on one mission, is one where the objectives were to verify 
the perfor man ce of the service propulsion subsystem. Several service propulsion 
subsystem burns of fixed time duration were planned for this purpose. Since obtaining 
specific velocity change is not a primary objective for these maneuvers, it is obvious 
that an increase in weight in one of the subsystems does not require additional propel- 
lant. Thus, the dry weight increase in a subsystem is the total weight increase of the 
spacecraft; that is, the trade-off factor for this maneuver is One. Another mission 
objective required that the Service Propulsion System provide a de-orbit impulse. 

This requires a fixed velocity increment. An increase in weight of one of the subsys- 
tems will result in lower accelerations, which in turn would result in a lower velocity 
if the burn time and thus the propellant were not increased. So, to maintain the mis- 
sion objective, propellant must be added, hence the trade-off factor is greater than 
One for this maneuver. The mathematical model selected for this last maneuver is 
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Table 3-3 

Apollo Launch Vehicle and Spacecraft Stages and Modules 
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Table 3-4 

Apollo Launch Vehicle Stage Functional Systems (206) and (504) 
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Interstage 

S-II 









































Table 3-5 

Apollo Spacecraft Module Functional System (504) 


Module 

Functional 

System 

LEM Ascent 

Structure 

X 

Stabilization and Control 

X 

Navigation and Guidance 

X 

Crew Provisions or Systems 

X 

Environmental Control System 

X 

Instrumentation 

X 

Electrical Power System 

X 

Propulsion System 

X 

Reaction Control System 

X 

Communications 

X 

Control and Displays 

X 

Landing Gear 


Earth Loading System 


Ballast 


Propellant - Useable 


Reaction Control (Useful Load) 


Electrical Power (Useful Load) 


Environmental Control (Useful Load) 


Main Propulsion (Useful Load) 


Scientific Equipment (Useful Load) 


Crew Systems (Useful Load) , 
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the ideal velocity equation. The trade-off factor, obtained by differentiating this equa- 
tion, is the ratio of the initial weight to final weight. 

In addition to the mission description, a description of the vehicle is required for the 
construction of math models to calculate predicted launch vehicle capability and space- 
craft weight. This description must include the breakdown to stages and modules and 
be related to the mission events. 

As development of the Apollo Program progresses, the mission details, the missions 
themselves, and the vehicle hardware are continually being modified or changed. This 
necessarily requires that the calculation procedure used to build up forecasted gross 
weights and launch vehicle capabilities be changed accordingly if the forecasts are to 
be accurate. An over- all surveillance of all available Apollo documentation, particu- 
larly revisions of those documents from which the mission and vehicle descriptions 
have been obtained, is continually being made so that the math models may be updated 
to include the latest data. 

3.7. 1.2 Buildup from Forecast Elements 

The major functional system data rather than stage weight, module weight, total pay- 
load, or launch vehicle capability was selected to be observed and trended with the 
trend forecast program. This level of system breakdown was selected because, first, 
different trend models are permitted for each functional system and the most appro- 
priate may be selected (a subjective decision made by managers experienced in the 
history of weight growth). Secondly, during the pre-trending analysis it is easier to 
assess changes in the data as random or non-random. Thirdly, it is easier to pin- 
point causes of weight problems when the analysis indicates a problem exists. Finally, 
experiences of Senior Weights Engineers associated with the development of these 
Weight/Performance Forecast Analysis techniques indicate that it is essential to ob- 
serve weight growth at no higher level than these major functional systems. Although 
it is desirable to trend functional systems from the viewpoint of the Forecast Program, 
it is not easy to evaluate the mission status from the forecasted weight of these many 
functional systems. Simple comparison of functional system weight forecasts to a con- 
trol value or limit would not indicate the existence or non-existence of mission capa- 
bility unless all or most of the forecasts were within their limits, an unlikely occurrence. 

To obtain the over-all mission picture from the forecasted weight of the major functional 
systems , a buildup of forecasted spacecraft weight and launch vehicle capability is necessary. 
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Also to be considered is the number of systems to be analyzed. There is an average 
of ten functional systems for about ten stages and modules, and for perhaps ten mis- 
sions, this gives a total of a thousand systems. Because functional system data is not 
always available for all missions and to reduce the amount of data to be trended, func- 
tional system growth forecasts are made only for selected, representative vehicles. 

For example, the Block n spacecraft functional systems are trended only for one mis- 
sion. The forecasted module inert weights for the spacecraft are obtained by slimming 
the weight forecasts, at the time of shipment, of the functional systems. The fore- 
casted module inert weights for other Block II spacecraft modules are obtained by as- 
suming their growth is the same over the period of time that they are being constructed. 
A sample calculation is shown in Figure 3-19 for the forecast analysis of the Block n 
command module. 

The control weights, established by program management from reference performance 
and trajectory requirements, represent the maximum weights for each individual stage 
or module. Should any stage forecasted weight exceed this control limit, the need is 
indicated for a required buy- off or reduction in weight, with some expenditure in money 
and time. Following the calculation of inert weights from the forecasted functional 
system weights, the total forecasted spacecraft module weights are calculated by ap- 
propriate use of trade-off factors. These calculations may be simple and straightfor- 
ward or, they may become somewhat involved, as in cases where the trade-off factors 
are not applied directly to the built-up inert weight, because of considering jettisoned 
and expendable items. The forecasted launch vehicle payload capability is calculated 
as shown in the sample calculation of Figure 3-20. The forecasted stage inert or burn- 
out weight is compared to the reported current weight, and the difference multiplied by 
the stage trade-off factor to yield the payload capability change resulting from weight 
change of that stage . These stage payload capability changes are summed for all of 
the stages of a launch vehicle and added to the current reported launch vehicle capa- 
bility to give the total forecasted launch vehicle capability. 

The buildup of the total spacecraft weight and launch vehicle capability described above 
is depicted in Figure 3-21. The first column illustrates the forecasted representative 
functional system weight, essentially obtained by summing the functional systems data, 
as described above. Finally, the last column shows the total launch vehicle capability 
or spacecraft weight which was built up from an appropriate combination of summing 
items from the previous column and multiplication with trade-off factors as described 
in preceding paragraphs. 
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Figure 3-19. Sample Trend Summing Program Output 


a. Command Module (CM) Predicted Inert Weight (from functional system 
predicted sums) 

10422 

b- Service Module (SM) Predicted Inert Weight (from functional system 
predicted sums) 

+9425 

c. Command/Service Module (CSM) Predicted Inert Weight (a + b) 

19847 

d. Lunar and Earth Orbit Expendables Weight (from current reported data) 

-485 

e. CSM Buyoff Weight at Transearth Injection (c - d) 

19362 

f. SM Injection Propellant Weight (Tradeoff Factors x buyoff weight) 

■ (0. 5130 x el ' ] 

+9933 

g. CSM Gross Weight at Transearth Injection (e + f) 

22925 

h. Lunar Excursion Module (LEM) Ascent Dry Weight at 
Buvoff (from functional system predicted sums) 

4334 


i. LEM Ascent Propellant Weight (trade-off factor x buyoff 
weight! (0. 4326 x hi 

+1875 


j. LEM Gross Weight at Liftoff (h + i) 

6209 

k. Weight of Items Jettisoned on Lunar Terrain (from current 
reported data) 

376 

1. LEM Dry Weight at Touchdown (from functional system 
predicted sums) 

+3472 

m. LEM Lunar Landing Weight (u + k + 1) 

10057 

n. LEM Descent Propellant Weight (trade-off factor x buyoff) 
weight! 10.5540 x ml J 

5572 

o. LEM Total Weight at Separation (m + n) 

15629 

p. Crew and Equipment Weight (from current reported data) 

-436 

q. LEM Total Weight at Injection ( o - p) 

15193 

15193 

r. LEM Lunar Orbit Expendables Weight (from current reported data) 

+337 

s. Weight at Lunar Orbit Injection (g + q + r) 

44825 

t. Injection Propellant Weight (trade-off factor x buyoff weight) 
iO. 3962 x si ^ ’ 

+17760 

u. Gross Weight in Earth Orbit (s + t) 

62585 

_v. Miscellaneous Weight Left in Earth Orbit (d - r) 

148 

w. Spacecraft Adapter Weight (from functional system predicted sums) 

+3142 

x. Injected Weight Less Reserves (u + v + w) 

65875 

y. NASA Design Reserves Propellant Weight (from current reported data) 

+1000 

z. Total Spacecraft Injected Weight (x + y) 

66875 

aa. Launch Escape System (LES) Weight (from functional system predicted sums) 

+6521 

ab. Total Spacecraft Liftoff Weight (z + aa) 

73396 


Figure 3-20. Sample Calculation of Spacecraft Weight Buildup 
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The comparison of reported and predicted launch vehicle capability to spacecraft weight 
is shown clearly on curves of the form of Figure 3-22. The implications of these curves, 
particularly when the predicted launch vehicle capability drops below the spacecraft 
weight, is discussed in detail in the next section. 

3.7.2 INTERFACE COMPARISONS 

Spacecraft and launch vehicle forecasts , which are compared to control limits for the 
purpose of identifying potential problems and obtaining quantitative data for manage- 
ment decisions, can also be compared at the major interfaces for each vehicle. These 
comparisons are made in order to obtain an over-all perspective of the weight-to- 
performance status of the vehicle. 

In order to understand the logic behind interface comparisons it is valuable to first 
consider control limits. Control limits comprise weight and performance budgets 
which are integral and compatible with the mission requirements. Changes in the 
control limits require compensating changes in related vehicle and mission specifica- 
tions. Individual stage and module control limits axe inextricably interrelated through 
performance criteria. 

Control limits are established down to the stage and module level, the lowest level to 
which weight, propulsion capability, and mission requirements can be conveniently 
related. Control limits are also mission oriented; they are established at each mis- 
sion event at which a stage or module with propulsion capability is critical to mission 
success. Some stages or modules have more than one control limi t due to weight 
changes during the mission resulting from the transfer or ejection of material and to 
differing propulsion system relationships. 

The buildup of data for the interface charts is similarly mission oriented. For exam- 
ple, the data prepared for the launch vehicle payload capability and spacecraft weight 
comparisons is oriented to the earth orbit or translunar injection event. The launch 
vehicle payload capability is adjusted for the effect of lifting the launch escape system 
which is ejected before earth orbit injection and therefore is not included in the space- 
craft weight. 

It should be noted that in the buildup of data for the launch vehicle payload capability 
and spacecraft weight comparisons, compensating changes in the stage or module con- 
trol limits and related vehicle or mission specifications have been assumed. These 
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112 lbs/months average decrease 
in payload capability 



Figure 3-22. Comparison of Reported and Predicted L/V-S/C Capability versus Weight 



assumptions are inherent in the procedures of summing the stage and module data, 
irrespective of control limits. The interface comparisons assume that the control 
limit can be increased for a stage or module having excessive growth providing a com- 
pensating decrease is made in the control limit of a stage or module for which the re- 
ported or forecasted weight is under its control limit. Concurrent changes in vehicle 
or mission specifications, such as propellant loading or velocity schedule, are thereby 
presupposed. These assumed ’trade-offs" are included in the launch vehicle payload 
capability and spacecraft weight comparisons in order to present the over-all picture, 
as illustrated in Figure 3-22. 

These types of trade-offs within the spacecraft or launch vehicle are resolved through 
use of trade-offs factors to reapportion the launch vehicle capability and spacecraft 
weight control limits. They are considered at the stage and module interface level. 
Examples of interface charts that depict the stage and module comparisons are shown 
in Figures 3-23 and 3-24. A problem is indicated when the reported or forecasted 
weight for a stage or module exceeds the control limit. This occurs in the charts when 
the reported or forecasted weight line ascends through the control limit line. 

In the launch vehicle to spacecraft interface charts the reported and forecasted payload 
capability of the launch vehicle and the reported and forecasted gross weight of the 
spacecraft are compared with their respective control limits and with each other. 
Comparison to each other identifies problems of injecting the spacecraft into earth or 
translunar orbit. Problems are indicated when the spacecraft reported or forecasted 
weight exceeds the launch vehicle forecasted capability curve, as shown in Figure 3-22. 
Beyond the intersection of the curve the spacecraft weight exceeds the launch vehicle's 
capability to carry it. However, this problem is preceded by the indication of internal 
problems within the spacecraft or launch vehicle evidenced by the respective curves 
exceeding their control limits. The spacecraft exceeds its control limit when it as- 
cends through its control line on the chart; and the launch vehicle capability exceeds 
its control limit when it descends through the control l imi t line on the char t. The 
interface charts presented in the Forecast Analysis Status Transmittal Report do not 
depict the problem of the spacecraft weight exceeding the launch vehicle capability but 
indicate the existence of the internal problems within the spacecraft or launch vehicle 
by a buy-off notation on the chart. 

Problems identified in the individual spacecraft module comparisons are diffi cult to 
assess relative to their status with other modules which are related by common 
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propulsion system requirements. This is particularly true of the Lunar Orbit Rendez- 
vous (LOR) mission vehicle because of the complex interrelations required to conduct 
this mission. Apportionment and the subsequent specification of weight and propulsion 
requirements are necessary in order to meet the spacecraft's mission requirements 
and yet not exceed the launch vehicle's capabilities. 

As another example, tank capacities are sized close to their specified requirements 
to minimize weight. These in turn impose physical restraints on weight growths and 
"trade-offs. " Therefore, charts which present the interface relationships of modules 
with common propulsion system requirements are prepared for the LOR mission space- 
craft at the translunar orbit mission event. The comparisons are referenced at the 
translunar orbit mission event although it would be more logical to orient the interface 
chart for the LEM modules to the lunar descent event. The comparisons were ref- 
erenced at the translunar orbit mission event in order to maintain correlation of values 
with the spacecraft weight to launch vehicle capability interface comparison. These 
charts are shown in Figures 3-23 and 3-24. The figures present the following comparisons: 

a. Figure 3-23 shows the combined reported and forecasted weight with fuel 
for the command module , service module , and adapter , and the total re- 
ported and forecasted weight with fuel for the lunar excursion module as 
compared to each other and to their appropriate control limits. Also, 
included on the same chart , the fuel required for service module propul- 
sion is compared to the service module tank capacity. 

b. Figure 3-24 shows a comparison, like the preceding one, of the weight of 
the lunar excursion module ascent stage and lunar excursion module de- 
scent stage with fuel; and the fuel requirements for LEM descent stage 
propulsion and LEM ascent stage propulsion to their respective tank 
capacities. 

3.7.3 WEIGHT /PERFORMANCE TRADE-OFFS 
3. 7. 3.1 Introduction 

Weight/performance trade-off factors are the means by which space vehicle stages and 
modules can be compared on a common, consistent, and meaningful basis. These fac- 
tors relate the change in weight of a given stage or module at any point in the mission 
to the change in weight of any other stage or module at any point in the mission. Using 
these trade-off factors , all weight deviations from control values are compared in 
terms of a single measure. 
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Figure 3-23. CSM/LEM Weight/Performance Constraint Interface 







3. 7. 3. 2 Determination of Trade-off Factors 


Trade-off factors are determined by varying one weight or performance parameter and 
observing its effect on other parameters of interest. The trade-off factor is then the 
ratio of the observed change to the imposed change. 

This measure in the LOR mission launch vehicle is equivalent to pounds of payload at 
(just after) translunar injection. For example , if a launch vehicle stage has a dry 
weight/ payload , capability trade-off factor of 10.0, it means that a weight change of 
10 pounds in that launch vehicle stage will result in a 1 . 0-pound change in payload just 
after injection, based on a postulated mission run with control weights. 

Therefore , a forecasted weight at launch showing 500 pounds overage in that launch 
vehicle stage represents a 50-pound decrease in payload capability. Assuming another 
stage of the same launch vehicle has a trade-off factor of 3.0 and a forecasted weight 
margin (negative overage) of 210 pounds it can contribute a payload capability margin 
of 70 pounds. The net effect of the off-control value forecasted launch weights of these 
two launch vehicle stages is then a 20-pound margin (over control value) in payload 
capability. 

The forecasted spacecraft (payload) weight at launch must be less than or equal to the 
forecasted payload capability at launch. A spacecraft module trade-off factor of 5.0 on 
a given spacecraft module for example, indicates that each one- pound change in the dry 
weight of this module results in a 5-pound change in the spacecraft weight required at 
translunar injection to accomplish the mission (including control performance and 
reserves). 

3.7.4 PROBABLE ERROR 

Probable error, as used in Forecast Analysis, is the numerical difference between the 
upper confidence limit and the forecasted weight at the shipping time. Probable errors 
are used as a simple indication of the degree of accuracy of the forecasted values , 
rather than confidence limits which are more difficult to portray in tabulated result form. 

The confidence limits, used in calculating the probable errors, are determined by the 
computer programs for calculating the forecast trend of the functional system data. In 
these computer programs the expected observations are assumed to be normally distri- 
buted about the prediction line, with a standard deviation <r^. For cases where there is 
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a large sample, say n > 10, it is a reasonable approximation to let probable error, 
PE, be: 

PE = c CT t 

where c is assumed to be a constant, equal to 1.645 for an infinitely large sample. 


The probable errors of the stages and modules and over-all spacecraft and launch ve- 
hicle are "built up" from the functional systems in a manner similar to the ,r buildup" 
of the forecasted weights as discussed in paragraph 3.7. The functional system fore- 
casted weights are added numerically to produce stage or module weights as follows : 


W 

stage or 
module inert 


FS.,^ + FS 2 + FS 3 + . . . 


where W is the composite weight and FS designates functional system weight. Assum- 
ing that the functional systems are uncorrelated ; * the weight standard deviation °w ls: 


W 


stage or 
module inert 


■ f 


FS, 


+ a. 


FS, 


+ a. 


FS, 


where is the composite standard deviation and a^g the individual functional system 
standard deviations. Substituting and cancelling the constant c, the composite probable 
error of the stage or module is the square root of the sum of the squares of the prob- 
able errors of the functional systems as follows: 


PE 


stage or 
module inert 


PE 


FS, 


+ PE 


FS, 


+ PE 


FS, 


As discussed in paragraph 3.7. 2, there are many functional systems to be monitored 
in the Apollo Program for the many mission vehicles with a single set of functional 
system data frequently used to forecast several like vehicle stages or modules. The 
ship date for the specific vehicle under consideration is used in the extrapolation of 
the functional system data. 


♦Uncorrelatedness, as stated in Reference 26 is satisfied when the covariance is zero, 
or the expected value of the product of any two functional systems is equal to the ex- 
pected value of the first times the expected value of the second. 
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Typical results of the computer analysis of the probable error for a series of wimila-r 
modules is shown in Figure 3-25. Functional systems of the modules were identified 
by code numbers shown to the left, a typical mission with ship date shown at the top 
and the probable errors and probable error squared in the two columns. The number 
at the bottom, i.e. , 33.57 is the square root of the sum of the squares and is the prob- 
able error for the total module. 

The 'buildup” of stage and module, and over- all spacecraft and launch vehicle forecast 
data is mission oriented as discussed in paragraph 3.7. Probable error for the over- 
all forecasts are also mission oriented and 'built up" in a similar fashion. Trade-off 
factors are applied in the calculation of the probable error of the spacecraft or launch 
vehicle weights with each of the modules or stage inert weight probable error values 
multiplied by an appropriate trade-off factor to account for the mission propellants and 
other performance requirements. 

The total weight is the sum of the products of the stage or module weights W x , W 2 , 
etc., andtheir appropriate trade-off factors F , F^, etc. , as follows: 

^total FjW i moc j u i e + F 2 W 2 jjyxjuig - +F 3^3 module + • • • 

or stage or stage or stage 

Assuming uncorrelatedness between stage or module growths and assuming the trade- 
off factors F x , F 2 , etc. , are constant, the over- all probable errors as well as prob- 
able errors associated with gross stage are calculated as follows: 


PE spacecraft or ' = ^ <*i PE / + < F 2 PE / + ( F 3 PE / + • • • 
launch vehicle 

where PE l5 PE 2 , etc., are the individual module or stage probable errors. 

The assumption that the factors (F) above are constant is equivalent to the assumption 
that the spacecraft or launch vehicle probable errors are due totally to changes in the 
module or stage inert weights and that there are no tolerances on the trade-off factors. 
The factors are used for calculating weights of propellants; therefore, errors in fore- 
casting propellant loading or utilization are not included in this analysis at this time . 
The influence of this assumption should be checked in more detail to evaluate the sig- 
nificance on the over-all results. 
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Figure 3-25. Module Probable Error Analysis 


Typical results of the computer analysis of a spacecraft are shown in Figure 3-26. 

The column of functional code numbers represents those major modules assembled to 
form the spacecraft. The next two columns illustrate typical trade-off factors and 
probable errors calculated for the modules. The spacecraft functional system number 
of 33.57 in Figure 3-25 now appears on the second row of the final calculation times 
1.66 or 55.67. The total spacecraft probable error is then calculated at 166.84. 

The relationship and use of probable error are further described in Appendix B of 
Book n. The details of the computer program to process the probable error calcula- 
tions is contained in the User's Guide in Appendix E of Book n. 

3.8 FORECASTS 

3. 8. 1 DEFICIENCIES AND BUY-OFFS 

In terms of over-all mission, the launch capability of the launch vehicle determines 
the control weight limit of the spacecraft to be launched. When the forecasted or ac- 
tual weight of the spacecraft exceeds the weight control limit, then a deficiency exists. 
Deficiencies are necessary buy-offs expressed in terms of total spacecraft weight or 
launch vehicle capability. They may vary from extremely difficult to relatively easy 
to correct. 

Figure 3-27 shows an example of spacecraft weight plotted against time to shipment. 

It can be seen that the observed rate of growth is fast approaching the control weight 
limit. 

The trend forecast line indicates that the control limit weight will be exceeded and that 
by the shipment date deficiencies will exist which will require a AW buy-off in space- 
craft weight or an increase in launch vehicle capability to offset the increased space- 
craft weight. 

As an example of calculation of buyoffs and deficiencies, let us suppose that the space- 
craft's lunar excursion module has an inert weight of 5100 pounds, and the control 
limit for the inert weight is 5000 pounds, giving a required inert weight buy-off of 
100 pounds. However, the 100 pounds excess inert weight may require an additional 
100 pounds of propellant to lift the module from the moon's surface back into l una r 
orbit. Furthermore, the additional 200 pounds of ascent module gross weight (100 lbs. 
inert +100 lbs. propellant) may require an additional 200 pounds of propellant to 
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Figure 3-26. Spacecraft Probable Error Analysis 




Time 


Figure 3-27. Forecast Buy-off at Shipment Date 


control the descent to the lunar surface. The penalty for 100 pounds of inert weight on 
the lunar excursion module is therefore a penalty of 400 pounds on the total LEM. This 
400 pounds is called the deficiency of the LEM. 


To re-numerate in tabular form: 


5100 - 5000 lb 


+100 lb LEM - Ascent Inert Buy-off 

+100 lb Propellant 

+200 lb LEM - Ascent Gross Buy-off 

+200 lb LEM - Descent Gross Buy-off (Propellant) 

-)400 lb LEM - Ascent Deficiency 


The deficiency, then, includes the inert weight buy-off plus needed additional propel- 
lant, wherever located. In the example given, the total 400 pounds ascent deficiency 
is comprised of 200 pounds of gross weight buy-off in the ascent module plus 200 pounds 
of gross weight buy-off in the descent module. 


In a similar manner, a 100-pound inert weight buy-off in a launch vehicle stage will 
require additional propellant to accomplish the mission. In the case of the launch ve- 
hicle , additional weight reduces the capability of the vehicle to carry the required pay- 
load. The deficiency for any stage is given in terms of reduced payload capability. 


3-49 



Experience shows that determination of buy-offs and deficiencies is an effective method 
of indicating and forecasting problems. 

However, merely to indicate that a stage or module is currently 100 pounds over its 
control limit and that therefore a problem exists or that there is a current required 
buy-off, still leaves a lot of questions unanswered. For instance, how difficult is the 
problem to correct? What is the growth rate? If the system is currently over its con- 
trol limit and the weight history shows a downward trend it may present less of a prob- 
lem than a system currently under its control weight but growing at an excessive rate. 
Where is the best place to look for a possible solution? Will there be a schedule slip 
involved? These questions and many others must be answered before an intelligent 
decision can be made. The next step of management decision process is the establish- 
ment of the criticality of making a decision. 

3.8.2 CRITICALITIES 

Apollo mission weight and performance weaknesses are referred to as the "Decision 
Relevancy Program. " The end results of this program are referred to as "criticali- 
ties. " One may speak of the "criticality" rating of an entire mission or any component 
of that mission, such as an individual stage or module. 

3. 8. 2.1 General Considerations 

After obtaining current status and forecasting vehicle weight and performance data, it 
is necessary to determine sources of potential problem areas and to obtain a "feel" 
for the criticality of foreseeable program weaknesses. Current and forecasted values 
for each launch vehicle, spacecraft, and their corresponding stages and modules can 
be compared with control values, and forecasted deficiencies can be obtained. To cor- 
rect forecasted deficiencies, however, may be extremely difficult or relatively easy. 
The deficiencies may involve major program decisions or simple corrective action 
that can be handled in the normal design cycle. It has been necessary, therefore, to 
develop an index of criticality which will emphasize the relative degree of seriousness 
of the problem. 

3. 8. 2.2 Ground Rules 

In developing such an index, the index model will, at best, be a guide based on arbi- 
trary ranges set by experience and engineering judgment. Although the gradation of 
criticality is established arbitrarily, the procedure should yield results on a consis- 
tent basis each time the model is used. 
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The model must consider current values as well as forecasts, since current values 
define the base from which actions must be taken. If the current base is below the 
assigned control value, the action may be one of "holding the line," i.e. , tighter 
management control. If the base is already above the control, a more serious prob- 
lem exists in taking corrective actions, even with the same forecasted deficiencies. 

In weighing the influence of current and forecasted values, it is necessary to consider 
the effect of program time. Early in a program, current values are largely estimated 
and so may contain m a ny errors. In this early phase, changes from month to month 
are extremely important because they indicate the direction that is being taken. T.at*> 
in the program, current values have become more accurate and the dominating influ- 
ence. The model, in addition, must consider that for the over-all mission the magni- 
tude of a problem may be measured by the performance margin between launch vehicle 
payload capability and spacecraft weight, that is, one performance parameter exceed- 
ing its control may be compensated by the other parameter being below its control. 

In summary, the model developed considers: 

a. Launch vehicle capability versus control value. 

b. Payload weight versus control value. 

c. Launch vehicle capability versus payload weight. 

d. Stage or module weight versus control value. 

e. Relative importance of current reported values versus forecasts as a 
function of program phase time. 

3. 8. 2.3 Method 

In order that consistency of results can be obtained, a mathematical model was devel- 
oped and programmed for computer application. This model then determines critical- 
ity for: 

a. Over-all mission. 

b. Spacecraft. 

c . Launch vehicle . 

d. Spacecraft modules. 

e. Launch vehicle stages. 

The gradations used are: 

a. Critical. 

b. Major weakness. 
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c. Minor weakness. 

d. Good shape. 

Table 3-6 presents the digital codes arbitrarily assigned to measure the criticality 
for launch vehicle capability, spacecraft weight, module or stage weight. These codes 
were selected by past experience and engineering judgment. In general, the digital 
codes represent a range of variation from control values. 

Figure 3-28 represents the influence of program time on the value placed on current 
reported values and forecasts. 

Figure 3-29 is used to quantize the over-all criticality. 

The basic steps for determining over-all mission criticality are as follows: 

a. Determine the following ratios: 

(1) L/V Capability (current) 

Control Value 

(2) L/V Capability (forecasted) 

Control Value 

(3) Control Value (current) 

Spacecraft Weight 

(4) Control Value (forecasted) 

Spacecraft Weight 

(5) L/V Capability (current) 

Spacecraft Weight 

(6) L/V Capability (forecasted) 

Spacecraft Weight 

b. Assign digital codes from Table 3-6 for each of the above ratios. 

c. Select time weighting factors from Figure 3-28. 

d. Determine over-all mission criticality index (Cl). 

_ Digital Codes X Corresponding Weighting Factors 
CI " 12 

e. Select over-all criticality from Figure 3-29. 

Indices are determined in a similar manner for each launch vehicle and its stages and 
each spacecraft and its modules. 
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Table 3-6 

Digital Code Criticality Assignments 


Stage or Module Weight versus Control 

Ratio 

Digital Code 

Gradation 

From 0.00 to 0.92 

1 

Critical 

From 0. 92 to 0. 96 

2 

Major Weakness 

From 0. 96 to 0. 98 

3 

Minor Weakness 

From 0. 98 to above 

4 

Good Shape 

Launch Vehicle Capability versus Control or Control versus Spacecraft Weight 

Ratio 

Digital Code 

Gradation 

From 0.00 to 0. 95 

1 

Critical 

From 0. 95 to 0. 98 

2 

Major Weakness 

From 0. 98 to 0. 99 

3 

Minor Weakness 

From 0. 99 to above 

4 

Good Shape 

Launch Vehicle Capability versus Spacecraft Weight 

Ratio 

Digital Code 

Gradation 

From 0.00 to 0. 95 

1 

Critical 

From 0. 95 to 0. 98 

2 

Major Weakness 

From 0. 98 to 1. 00 

3 

Minor Weakness 

From 1.00 to above 

4 

Good Shape 
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Months to Ship 

Figure 3-28. Program Phas^Time Weighting Factors 




Value F] 


Minor Weakness 



Figure 3-29. Over-all Criticality Determination 


3.9 APPRAISALS 


3.9.1 COST /WEIGHT RELATIONSHIPS 

3. 9. 1.1 General 

Cost is a governing parameter determining the amount of technical development and 
production work that can be done on a particular space program. Since costs have a 
direct bearing on what can be done , they must be considered in all trade-offs made 
during the engineering development effort. An effective and meaningful relationship 
between cost and program control is essential for program managerial trade-offs. 


To arrive at cost estimates some arbitrary assumptions have to be made and available 
data extrapolated. Cost information is usually available from previous similar pro- 
grams, the best starting point for cost analysis. Obviously, this extrapolation of old 
data must be coupled with engineering judgment and analysis in new cost estimations. 

The usual procedure is to estimate costs for required buy-offs through a cost /weight 
relationship on a per pound basis. Admittedly, the procedure yields only a rough es- 
timate, but if used with caution and the full awareness of the assumptions involved, 
the estimation can serve as a useful forecast of trade-off values. 

Costs are aff ected by many parameters, some more significantly than others. An 
averaging technique can be used to smooth out the peaks and valleys which tend to give 
erratic answers. Obviously, it is possible to miss a significant peak in the cost curve, 
but engineering judgment and specific data on a particular application lend validity to 
the cost estimating technique. Once cost problem areas have been identified, the de- 
tailed analysis which follows becomes the prime consideration in the problem solution. 

Weight is a useful relating index of cost. Correlations between weight and cost are 
well established from known types of equipment, such as aircraft, launch vehicles, 
machine tools, etc. In other areas, such as electronic equipment, weight bears little 
relationship to cost because of today 1 s emphasis on miniaturization which in fact 
creates an inverse cost ratio to weight. 

3. 9. 1.2 Over-all Cost 

Fortunately, for reasonable results, cost/weight trade-offs need be considered only at 
the major stage or module level where detail inequalities tend to be masked by an 


3-56 


over-all averaging effect. In addition, cost/weight relationships can be updated from 
actual cost figures for the specific program of interest. 

In general, cost factors can be grouped into five kinds of elements, development costs, 
production costs, test costs, facility costs, and operational costs. For purposes of 
developing cost trade-off factors, development and test costs are lumped into R&D 
costs, as shown in Table- 3-7. Non-direct portions of facility cost, operation cost, 
and other non-direct costs are not computed since these costs are assumed unrelated 
to the trade-off factors sought for cost/weight. 

Monthly production costs can be determined from the cost and schedule data for the 
latter part of the program, during which time the research and development costs are 
assumed to be negligible compared to the production costs. 

From the data available for the first part of the program, the research and develop- 
ment costs are separated from the production costs by subtracting the production costs 
from the funding allocated for any given fiscal period. 

Research, development, and production costs are summed to obtain the total costs of 
the indicated weight reductions. 

In calculating the additional cost due to a required buy-off, research and development 
costs are allocated on a per- pound basis, using an exponential distribution. This dis- 
tribution is obtained by assuming that the re-expenditure of the total research and de- 
velopment cost yields a 10-percent weight reduction; that is, in order to obtain a 100- 
pound buy-off, 1000 pounds of a stage/module must be redesigned. 

Production costs depend on the phase of production which a particular stage/module is 
in at the time of interest. A 10-percent yield is assumed again as a reasonable return 
from improved production methods for reduced weight. 

The research and development costs of removing a deficiency is assigned to the first 
vehicle in which it occurs even though the results of the research and development ef- 
fort may be realized on subsequent stages or modules. The production costs differ 
for specific stages (or modules) since not all stages are in the same production phase. 

To obtain the schedule change due to a required buy-off, the total cost of a specific 
buy-off is divided by the current monthly spending rate for the applicable stage or 
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Table 3-7 



S-IC 














module. This assumes that the current level of effort on a specific stage or module 
will be maintained during the implementation of required buy-offs. 

A sample calculation of this over-all cost estimating technique is illustrated in Fig- 
ure 3-30. For this example, a 500-pound weight reduction is assumed to be required 
in each of two vehicles, A and B. Vehicle A is 20 months along in production, vehicle 
B is 10 months into production. The necessary R&D costs are computed as indi cated 
in the upper left of Figure 3-30 as 4800 cost units regardless of whether one or two 
vehicles are involved. The production costs are computed in relation to the phase , or 
months progress in the production cycle. In the illustration, the costs of reducing 
weight are indicated as 30,000 and 15,000 for vehicles A or B. For weight reductions 
in both vehicles, costs are accumulative, or 45,000 cost units. The schedule slippage 
associated with these weight reductions are computed using the average rate of pro- 
gram expenditures as an adjustment factor. As indicated in Figure 3-30, the expected 
schedule slip is computed by dividing the total cost of weight reduction of average 
spending rate. 

Typical trade-off factors, shown as hypothetical values, are s ummar ized in Table 3-7 
for representative stages and modules. Since performance relationships are known 
between the various stages and modules , for ease of comparison weight can be ex- 
pressed in terms of equivalent spacecraft or launch vehicle weight. The numbers in 
Table 3-7 are directly comparable, showing the hypothetical relationship between cost 
and weight saving on various stages. The data provides a useful tool for management 
trade-offs between stages or modules for optimization analysis. 

For convenience, the trade-off factors can be shown in slide rule form, such as shown 
in Figure 3-31. Handy reference and rapid use of cost/weight factors can then be 
achieved by the various management groups concerned with program control. 

3. 9. 1.3 System Costs 

It is recognized that an accurate cost analysis must be performed on a system basis. 

To accomplish this refinement, a system cost model is used. 

This model allows, for example, structural system costs to be differentiated from nav- 
igation and guidance system costs. When considered on a per pound basis, the costs 
of these two systems are significantly different and must be considered separately. 
Vehicle costs are defined to the extent that the breakdown allows required buy-offs to 
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Estimate cost of a 500-pound weight reduction on sample vehicles A and B, which have 
similar components but fly on different missions. 


Production Costs 


R&D Costs 


_ w b D = Required R&D Costs 
D " aW ’ W = Req’d. Weight Red. 

a,b = Constants / 
Vehicles A and B / 


Vehicle A and B Production Cost 
Units/ Pounds- Month = 0.3 

Production Weight = 1Q x 500 = 500 o 
Requiring Redesign DOUnds 

(Assume 10% Yield) p0undS 


Vehicle A is 20 Months Into Production 

Vehicle A Production Cost = 20 x 0.30 
x 5000 = 30,000 Cost Units 


Vehicle B is 10 Months Into Production 

Vehicle B Production Cost = 10 x 0,50 
x 5000 = 16,000 Cost Units 


= 48,000 Cost Units 


Cost of Weight Reduction 


Vehicle A Only 

48,000 

30,000 

78,000 

3 

Vehicle B Only 

48,000 

15,000 

63,000 

2 

Vehicles A and B 

48,000 

45,000 

93,000 

3 


♦Schedule Change 

Current Vehicles A and B spending rate = 33,000 cost units/month. 

„ , . . .. Total Cost of Weight Reduction 

Schedule Change due to Weight Reduction = 33,000 Cost Units/ Month 


Figure 3-30. Estimated Over-all Costs - Sample Vehicles A and B 
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Launch Vehicle/Spacecral't Trade-off Selector 





be traded off with the lesser expensive systems and indicates dollar usage efficiency 
when a given required buy-off is being analyzed. 

The model illustrates particular system sensitivity to cost and is primarily intended 
for analyzing required buy-off costs in detail when specific problem areas occur or 
are forecasted. The model utilizes past program data to develop cost curves for each 
system. An example of a system cost calculation is illustrated in Figure 3-32. 

3.9.2 SCHEDULE /EVENT RELATIONSHIPS 

3. 9. 2.1 Program Status 

In order to determine whether a program is ahead of schedule , or behind schedule and 
how far, time-varying program parameters must be utilized. To augment cumulative 
cost and schedule data for purposes of evaluating program status, Estimated, Calcu- 
lated, and Actual weight percentages (E/C/A) are considered a related time-varying 
parameter useful as a program status barometer. On this basis an effective schedule 
forecast, i.e. , an indication of program status is made. 

An E/C/A schedule model is developed from available schedule data for each stage and 
module. This is done by making reasonable assumptions as to what the E/C/A data 
should be during various phases of development, production, and testing of each stage 
and module. This model then represents the forecast of the E/C/A data. 

Observed E/C/A data is reported monthly, by the appropriate contractors, on each 
functional system. The reporting of these data, along with the corresponding weight 
data, is a contractual requirement. The reported functional system E/C /A data are 
combined proportionately to obtain the reported E/C/A data for each stage and module. 

Eacn month reported E/C/A data for each stage and module is compared with the E/C/A 
schedule model and the forecasted schedule differences calculated to determine how far 
the program is ahead or behind of schedule. 

As an example, Figure 3-33 illustrates the monthly reported E/C/A data for sample 
vehicle A, plotted against the model for this particular vehicle. From this plot it can 
be seen that the model indicates vehicle A to be one month behind schedule (average 
status) . 
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Cost Units 


(Unit Cost) 



Weight Weight 



System Costs 


R&D 

(Cost Units) 

Production 
(Cost Units) 

Total 

(Cost Units) 

100 Pounds of System I 

300 

180 

480 

100 Pounds of System II 

100 

75 

175 


Figure 3-32. Estimated System Costs Sample Systems I and n 










Figure 3-33. Monthly Reported E/C/A Data for Sample Vehicle A 



3. 9. 2. 2 Use of E/C/A Values as Program Maturity Indices 

Once the model of E/C/A is established for each stage, module or functional system, 
tiie predicted portion of this model can be updated as additional monthly observations 
are made. These corrections can take two forms, first the adjustment of the schedule 
as noted above and secondly the reshaping of the forecast curves of E/C/A to better 
conform to trend patterns. 

These updated E/C/A curves in the forecast time domain can then be used to improve 
the slope of the weight prediction line. Here E/C/A values reflect program maturity 
and can be used as additional factors in shaping forecast line slope. For example, 
slope is assumed to reduce by a factor related to the completion of the component fab- 
rication and test, as evidenced by decreasing E and C and increasing A. The feedback 
of actual E/C/A values into this model provides a measure of self- correcting action 
since the shape of the E/C/A curves become more apparent as additional observations 
are made. 

3.9.3 RELIABILITY AVEIGHT RELATIONSHIPS 

3.9.3. 1 Object and Scope 

Techniques are required for accurately determining the effect of weight changes upon 
mission reliability. The weight changes considered will be limited to weight reductions 
of an already designed space vehicle and provide only a cursory examination of the sub- 
ject. The techniques discussed here are not allocation optimizing methods, which are 
far more complicated. 

3. 9. 3. 2 Definitions 

The term mission reliability is commonly used to denote the probability of mission 
success. However, mission reliability also includes the probability of crew safety. 
Calculation of the probability of crew safety causes many of the difficulties of compu- 
tation because of the many different mission aborts which must be considered. 

Weight changes, especially removal of redundant equipment, will affect probabilities 
in different ways . Therefore , both sets of inter actionsmust be considered and computed. 
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3. 9. 3.3 Weight Reduction Methods 

Weight reduction in a space vehicle can come about only by two methods: 

a. The removal of matter from the space vehicle. 

b. The substitution of lighter material for heavier material. 

The first of these (removal) can come about as a result of a number of operational de- 
cisions. The second (substitution) can occur among various elements of the vehicle. 

3. 9.3.4 The Effects of Weight Reduction on Reliability 

There are a number of interactions between weight (reductions) and the probability of 
mission success. Consider the immediate first order effects: 

a. The removal of redundant equipment will decrease the probability of 
mission success. 

b . The removal of a functional requirement and its associated equipment 
will increase the mission success probability. 

c. The use of existing equipment for other functions, or the substitution of 
a lighter system for a heavier one , may increase or decrease the mis- 
sion success probability. Each case must be analyzed separately. 

d. The elimination of non-essential equipment, waste product dumping, or 
removal of structural material, has no direct effect on mission success 
(assuming the structural reliability is not reduced). 

In order to analyze the effects of weight reduction on vehicle reliability, a function re- 
lating these parameters is required. The function most suited to this is the ratio of 
reliability decrement per pound of weight removed. This ratio can be calculated for 
the whole vehicle, or for a stage, module, or system as desired. Generally, a curve 
of this function will have a constantly increasing slope as shown in Figure 3-34. 

An equation for this curve may be established after a number of analyses have been 
made which will permit its construction from vehicle design data or, the curve can be 
constructed by individual computations of the function on a system-by- system basis. 

3. 9. 3. 5 Reliability Prediction Techniques 

3 . 9 . 3 . 5 . 1 Approximation Method 

The approximation technique is a fast, economical tool for obtaining an average ratio 
for the reliability decrement/ pound obtainable from redundancy removals. 
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Figure 3-34. Plot of Effects of Weight Reduction on Reliability 


While this ratio will not necessarily apply to any single system, the ratio and the total 
values obtained for weight reduction and reliability decrease can be used as a decision 
making tool for testing the feasibility of redundancy removal for a particular vehicle , 
stage, or system. Also comparisons between vehicles, stages, or systems can be 
made on this basis. 

The method is as follows: 

a. Given a reliability model, which has been solved for the mission suc- 
cess and crew safety probabilities, remove all redundant elements 
from the model. 

b. The remaining series model can be solved fairly quickly on a desk 
calculator and a new estimate of mission success obtained. 

c. The weight of the parts eliminated can be found and then an average 
ratio of reliability decrease/pound can be calculated. This calcula- 
tion can be performed for the whole vehicle, as well as by stage and 
module. This solution gives the limit of weight reduction by removal 
of redundant parts, and the change in mission success probability. 

d. The change in crew safety probability must be determined by means 
of a failure effects analysis of the reduced system. 
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3 . 9 . 3 . 5 . 2 Selective Redundancy Removal 

A simple technique for finding the change in reliability resulting from the removal of 
some redundancy in a given system is available from the results of the Simulation of 
Apollo Reliability (SOAR) computer runs. 

For example, assume that in a system that consists of three elements in parallel, it is 
desired to find the effect of eliminating element 3 on the mission 
reliability. Element 3 will be used only if elements 1 and 2 have 
failed. Examining the SOAR computer printouts, all Monte Carlo 
trials where elements 1 and 2 have failed, but where the mission 
succeeded because of element 3, are reclassified from mission 
success to mission failure. The new probabilities are then read- 
ily found by dividing the revised mission success count by the number of trials. 

As an example, consider a spacecraft electrical power system for which a computer 
run of 10,000 Monte Carlo trials had been made. The components selected for re- 
moval were one inverter, one re-entry battery and one fuel cell. The system con- 
tained three each of these components which were in various mission phases required 
to operate in one of three, two of three, or three of three configurations. 

In each case, upon removal of the third component, the reliability logic diagram was 
adjusted to indicate the required use of the remaining two components in either series 
(two of two) or parallel (one of two) configurations. 

The computer results were then examined to determine the changes in probability of 
mission success. 

The results are shown in Table 3-8. 

Table 3-8 

Sample Selective Redundancy Removal Calculation Results 


— 0-1 
— od- 


Component 

Approximate 
Component 
Wt. (lbs) 

Increase in 
Number of 
MS Failures 

Reduction in 
Mission Success 
Probability 

Reduction of 
Reliability 
Per Pound 

Fuel Cell 

100 

187 

0.0187 

0.000187 

Battery 

50 

16 

0.0016 

0.00032 

Inverter 

50 

210 

0.0210 

0.000420 
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Examination of these results reveals the following: 

a. For weight reduction up to 50 pounds - remove the battery. 

b. For weight reduction from 50 to 100 pounds - remove the fuel cell. 

c . For weight reduction from 100 to 150 pounds - remove the battery 
and fuel cell. 

d. For weight reduction over 150 pounds - remove all 3 components. 
The reliability reductions for multiple removal are: 




AR/lb 

Battery and inverter 

0.0223 

0.000223 

Battery and fuel cell 

0.0200 

0.000133 

Inverter and fuel cell 

0.0397 

0.000265 

All components 

0.0403 

0.000202 


3 . 9 . 3 . 5 . 4 Other Techniques 

There are many other techniques available for obtaining reliability estimates. Some 
methods which may be useful are: 

a. The minimum cut method. 

This method uses Boolean algebra to evaluate the reliability of com- 
plex networks, which are subject to configuration changes. 

b. The partial derivative method. 

This method assesses the effect of changes in component reliability 
on systems reliability. 

c . Correlation techniques . 

After a number of analyses have been made and a data bank of reli- 
ability/weight relationships has been built up, standard statistical 
regression and correlation techniques can be used to attempt to es- 
tablish a direct relationship between these factors. 

3. 9. 3. 6 Multiple Factor Relationships 

3. 9. 3. 6.1 Introduction 

Vehicle weight and reliability are also indirectly related through the factor of 
performance. 
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Schematically this can be shown as: 


A weight 


transfer 

A performance ^ 

transfer 

A reliability 

function 


function 



In a complex system the "transfer functions" may not be directly expressible as equa- 
tions, but they can represent the analyses necessary to relate the parameters. In the 
case of a manned vehicle- the "A performance” parameter can refer to hardware and/or 
crew performance. 


3. 9. 3.6. 2 Weight - Crew Performance - Reliability 

As stated in paragraph 3.5.4, removal of equipment not mission essential has no di- 
rect influence on reliability. Removal of such equipment may, however, affect reli- 
ability by affecting crew performance. The problem with changes of this type is that 
they are extremely difficult to quantize. Usually only a qualitative statement about 
their effect on reliability can be made. 


3. 9. 3. 6. 3 Weight - Vehicle Performance - Reliability 

If weight reduction causes a change in the weight/thrust ratio , the effect of perform- 
ance changes on reliability will be due to the changes in the length of the mission 
phases. For example, a heavier vehicle will require longer engine burn times to 
reach an injection point. This will result in a reduction in engine reliability and will 
also reduce the reliability of all other systems, due to their increased exposure to 
the high-stress environment existing during thrust. 

This change in performance must in turn be translated into a change in reliability. 

The analysis is extensive and expensive and should be made only when significant 
weight and performance changes have occurred. After a number of weight/performance 
reliability studies have been made, statistical techniques of multiple regression and 
covariance analysis can be used to discover relationships between the parameters use- 
ful in trend forecasts. 


The techniques of system optimization under one or more constraints are well described 
in the references listed in the bibliography. 
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. 3.10 ILLUSTRATIVE RESULTS USING HISTORICAL WEIGHT DATA 


3.10.1 VALIDATION BACKGROUND 

The validation of current weight performance prediction methods is important for 
three reasons. Perhaps the most important is the determination of their credibility 
under a variety of circumstances. This information leads directly to the second 
benefit - an indication of which combinations of available techniques when applied to 
sets of data exhibiting certain identifiable characteristics will provide the most ac- 
curate forecasts of future performance. Thirdly, the validation process provides 
significant guidance for the development of advanced prediction methods and automatic 
data handling techniques. 

The validation program described here was designed to provide this information in a 
comprehensive and objective manner. The approach taken consisted of: (a) defining 
specific objectives, (b) defining specific quantities to be measured and ways of meas- 
uring them and (c) developing a matrix of computer runs which when subjected to (b) 
will satisfy (a) . A special effort has been made to include only those analyses directly 
affecting the stated objectives. The analyses were further restricted by limiting their 
scope to include necessary and sufficient operations only since the number of potential 
variables of interest is large. 

The following goals of the validation effort are not independent but identify the three 
distinct types of information required: 

a. Determine the extent to which present prediction programs and methods 
provide accurate forecasts of future weight performance. 

b. Determine the sensitivity of the prediction techniques to variations in 
their manner of application, data characteristics, values of internal pro- 
gram constants, etc. 

c. Develop additional insight into and provide guidance for contemplated 
developments in advanced normalization techniques, trend mode selec- 
tion and prediction methods. 

Accomplishments of the stated goals require the orderly evaluation of the following 
items: 

a. Regression curve goodness -of-fit in the observation range. 

b. Forecast Accuracy (measured at end of observation range). 
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c. Effect of normalization 


(1) removal versus non-removal of non-random changes. 

(2) changes in definitions of non-random changes. 

(3) outlier removal versus no outlier removal. 

(4) changes in outlier limit boundaries. 

(5) multiple application of outlier removal option. 

d. Effect of variation in internal constants (R's, a, y, etc.). 

e. Effect of time conversion (real and project time) . 

f. General applicability of trend-of-the-trends. 

g. Correlation between trend-of-the-trends results and (a). 

h. Convergence of repeating mode analysis (RMA) results: 

(1) at last observation point. 

(2) at shipping point. 

(3) correlation between (1) and (2). 

i. Effect of the number of RMA passes per data set. 

j. Effect of normalization on RMA results. 

k. Effect of trend mode on RMA results. 

l. Consistency between suboptimization and optimization (do optimum func- 
tional subsystem results necessarily sum to optimum stage results?). 

m. Value of probable error prediction. 

n. Validity of confidence intervals. 

o. Benefit of running RMA in reverse. 

Careful inspection of this list will indicate that there are numerous compound over- 
laps - that is groups of two or more of the items can be accounted for in individual 
analyses. Recognition of this fact is the basis for streamlining an otherwise gigantic 
and unwieldy program. 

Performance measurement is a critical element in any evaluation procedure. Meas- 
urement means and standards must be clearly defined and adhered to so that objective 
and consistent evaluations will be obtained. 

In the present case, the majority of useful information will be derived from compara- 
tive analyses or relative performance. This introduces a subjective element which 
must be controlled to whatever degree possible by the use of analytic measures. The 
concept, then, is to make maximum use of such mathematical tests and measures as are 
available to provide data for the comparative analyses. Those which have been defined to 
date are: 
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a. Mean square error - a simple and direct analytical means for measuring 
the degree to which regression curves represent the subject data. 

b. Ratio of mean square successive difference and estimate of variance - an 
analytical comparison of two quantities each of those expected values is 
the distribution variance; another means of measuring regression curve 
fit. 

c. Differential between the last observed data point and its predicted 
values - a measure of prediction accuracy (not applicable to the Fourier 
Analysis model) . 

d. Convergence/divergence of RMA results - decay rate in convergent 
cases will serve as an indicator of relative value. 

e. Variantions in trend line slopes. 

The actual validation program recognizes the inherent relationships among the items 
to be evaluated. It assures that the maximum information will be obtained from a 
given set of computer results and analyses. The net effect is the definition of an effi- 
cient program plan containing a minimum of unnecessary effort. 

It should be realized, however, that even with a well defined plan it is difficult to 
foresee exactly how the program's progress will unfold. New relationships will be 
discovered, some steps will be found unnecessary, additional analysis will be called 
for in some area, etc. Therefore, even though the program of computer runs de- 
veloped is felt to be an efficient and sufficient one, it was implemented with the under- 
standing that further reductions or additions may be identified as the program pro- 
gresses. 

An examination of the list on pages 3. 71 and 3. 72 shows that one way of grouping the 
items is as follows: 

Group I: Those items involving pre-trending operations on the data - con- 

sisting of a, b, c, e, 1, m. 

Group H: Those items involving Repeating Mode Analyses and Trend-of-the- 

trends - consisting of a, b, f, g, h, i, j, k, o. 

Group EH: Those items involving variations within the trending programs 
themselves consisting of a, b, d. 

It will be noted that items a and b are common to all three groups. These two items 
enjoy a double role - being both measures and things to be evaluated. Their role as 
measures causes the triple inclusion here. 
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The techniques for Forecast Analysis have been tested against historical weight data 
from aerospace programs and prior weight data on the Apollo Program to see how 
well they would have worked if used on those actual programs. 

Representative programs used for validation were: 

a. Saturn I Launch Vehicle 

b. Nimbus Spacecraft 

c. Mercury Spacecraft 

d. Saturn V Launch Vehicle 

e. Saturn IB Launch Vehicle 

f. Apollo Spacecraft 

In the first three programs listed above, weight data is available, although somewhat 
inconsistent. On the latter three programs, good weight data is available, but the end 
results are unknown at the date of writing. 

3. 10. 3 HOW ACCURATE SHOULD A PREDICTION BE? 

An interesting question which is raised after some study of validation is, "How accur- 
ate should the forecasts be?" Or more specifically, "Should the actual observed launch 
weight be forecast one to two years in advance?" The answer to this question is depen- 
dent on the expected "elbow room" allowed between current weight and upper control 
weight values. If the forecast growth 
carries the expected weight past the 
control limit, it is natural to assume 
that certain corrective action will be 
taken. In fact, this action identified 
as "required buy-offs "are the essen- 
tial identifications of Forecast Analy- 
sis and it is presupposed that neces- 
sary managerial action will be taken 
and the final launch weight will be 
brought below the control limit. 

The benefits from Forecast Analysis come from early recognition of problems so that 
corrective action may be made in timely, economic fashion. 

On the other hand, if there is sufficient "elbow room" between forecasted weight and 
control weight values, the natural growth would be expected to continue. 
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Thus, the word "error" is introduced in a mathematical sense only, with recognition 
that the final weight will be frequently different than the forecasted weight because of 
incorporation of buy-offs. This leads to the interesting conclusion that it may be self- 
defeating to attempt to improve forecast accuracy since in some cases it is not reason- 
able to expect final weight to be equal to the forecasted weight. 

It is sometimes argued that forecasts should recognize expected incorporation of future 
buy-offs and be adjusted accordingly. Such an argument neglects the need for clear 
presentation to management of the required magnitude of corrective action, and there- 
fore Forecast Analysis validation efforts are caught in a dilemma. "Should manage- 
ment corrective action be presupposed and thus run the risk of failing to emphasize the 
urgency of such action ?" 

The alternative which is selected here is to present the "natural" growth expectations, 
with the forecasted buy-offs incorporated only when actually authorized or actually 
made. Studies for validation should consider methods by which data can be "prenorma- 
lized" so as to permit study on a more consistent basis. Prenormalization is used to 
describe the preremoval of all buy-offs and is possible by using existing repeating- 
mode programs which permit a more consistent comparison of all data on the same ba- 
sis. With prenormalization, forecasts should be close to actual measured values, but 
this can be done only after the final weight is observed. 

3.10.4 A MORE DETAILED STUDY OF S-IV STAGE OF SA-5 VEHICLE 

Application of prenormalization and other techniques was studied in a more detailed 
analysis of the Saturn I, SA-5 vehicle, S-IV stage. Figures 3-36 through 3-42 show 
the results of analysis with data assessed as it would be during the program, as well 
as for prenormalized data. 

The curves are the percent differences of the forecasted and actual data at the forecast 
time. Positive values are percentages by which the forecasted values have exceeded 
the actual value. The data of the "normalized" curve was processed in a form that 
would occur during a program. The "prenormalized" data was corrected for all non- 
random changes prior to extrapolation. 

An illustration of benefits of prenormalization can be seen in Figure 3-36 where a large 
increase in weight occurred in the Equipment and Instrumentation System between Jan- 
uary 1963 and June 1963. The large difference between the normalized and prenormal- 
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Figure 3- 37. Equipment and Instrumentation System Comparison of 
Computer Programs Prenormalized Data 





Percent Error 



Figure 3-39. Structure System Comparison of Computer Programs 
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Figure 3-40. Propulsion System Comparison of Computer Programs 
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Figure 3-41. S-IV Stage Separation Weight (Sum of Systems) Trend Selection 
Comparison Prenormalized Data 
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Figure 3-42. S-IV Stage Separation Weight (Sum of Systems) Comparison of 
Computer Programs Prenormalized Data 

ized curves for the system are as expected and point up the need for careful, continued 
study of normalization and forecast adjustment for management actions. 

The influence of biasing is shown in Figure 3-37. Forecast data is biased in order to 
give a positive response to changes and improve forecasts in the near future. Normali- 
zation also has the effect of accelerating the influence of changes on forecasts. Nor- 
malization is primarily incorporated to maintain a correlation between the system and 
and the mathematical model as shown in the preceding paragraph. In the beginning of 
the program, normalization has a greater effect on the forecasts than does biasing. 
Biasing becomes effective during the program and more so toward the end of the pro- 
gram. This is to be expected because normalization of a change has a salutory effect 
on the slope of the trend line early in the program as small unnormalized changes 
would have an appreciable adverse effect due to the small sample size. In Figure 3-37 
the dotted line represents data that has been biased only, the dashed line is data that 
has been normalized only and solid line is data that has been both normalized and 
biased. The alternate dash and dotted line is data that has been neither biased nor 



normalized. It will be noted that the dashed line (normalized data) is nearly coincident 
with the solid line (biased and normalized data) at the beginning, and that the dotted 
line (biased data) is nearer the solid line toward the end. 

Comparison of the computer program forecasts are shown for the Saturn I, SA-5, S-IV 
stage and systems in Figures 3-38 through 3-42. The comparative accuracy of each 
program is evident. 

Since Forecast Analysis is an evaluating process, it is essential that validation studies 
be conducted parallel with its development and application. The results can be signi- 
ficant in establishing the credibility of the techniques developed and in generating guide- 
lines for future development. 

3.10.5 ANALYSIS OF REPRESENTATIVE APOLLO SYSTEMS 

Six cases were selected from the weight program. The only requirements were that 
they have on the order of 20 points and that they be "typical" - that is, that the data 
variations are no smoother than one would ordinarily expect. 

Each case was treated by each of the four prediction models and the Repeating Mode 
Analysis (RMA) routine. Thus, 24 case/model combinations were processed by RMA. 
The first prediction, in every instance, was made using the first 8 points in the set. 

One additional point, in chronological order, was added during each pass through RMA 
until the last prediction was made using the complete data set less the last point. All 
6 cases used the outlier removal option while 2 of them were also run without it, for 
the sake of comparison. 

The predictions themselves extended only to the last observation point. Thus, all pre- 
dictions and analyses are based on an artificial prediction range, wholly contained with- 
in the actual observation range. The change removals were made only as they occurred, 
that is, the data were not "prenormalized. " 

Three types of error were formed and recorded. The Type I error is the sum of the 
squares of the differences between the trend line and the normalized data points used 
to generate it. The Type II error is the Type I error plus the sum of the squares of the 
differences between the prediction values and the corresponding un-normalized obser- 
vation values. The Type in error, computed once for each RMA sequence, is the sum 
of the squares of the differences between the predicted and observed values of the last 
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. observed point. The Type I error indicates the goodness-of-fit and the Type m error 
is a measure of the average ability to predict the last observed value. (In practice, 
these two measures never coincided. ) Upper confidence limits were also printed out 
in the hopes that some useful correlation might be uncovered. As it turned out, all 
observations were either outside the limits or well inside , suggesting that a further 
evaluation of the confidence limits themselves is required. 

In developing the results listed in Chapter 2, the RMA outputs were examined in 
the following ways: 

a. The prediction error at the last observation point, for each model, was 
plotted versus the number of points used in the prediction. 

b. The observed values, both pre- and post-normalization, were plotted 
versus the observation time. 

c. The prediction error at the observation point immediately following the 
last point used for the prediction, both pre- and post-normalization, was 
plotted versus the number of points used in the prediction. 

d. Tabular comparisons of the 3 error types. 

e. Tabular listing of the prediction errors at each observation point versus 
the number of points used in the prediction. 

f. Tabular comparisons of the prediction errors at the last observation point 
expressed as percentages* 

A typical result is presented in Figure 3-43. 

It should be stressed that the following discussions are based on the analysis of a small 
sample. Therefore, conclusions about the predictions themselves are considered as 
tentative only. The reader's interest should be concentrated upon the measuring tech- 
niques and the nature of the information they provide. 

Figure 3-44 presents the results of applying the repeating mode routine to one set of 
data set, consisting of 30 points. The first pass predicted the value of the 30th point 
based on the first 8 points. The second predicted the value of the 30th point using the 
first 9 points, and so on. These predicted values, using all four prediction models, 
are shown here plotted versus the number of points used and are compared with the 
actual or observed value of the 30th point. The important thing to note is the conver- 
gence on the observed value as time passes. 
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Figure 3-43. Prediction Error at Last Observation Point of a Representative Case 











•Also indicated in Figure 3-43 are the times at which gross errors and non- random 
changes were removed from the original data. Further insight is provided by Table 3-9 
which tabulates the prediction accuracy and the magnitude of the non-random changes 
applied. Note the 11-percent improvement in accuracy when the 14-percent change is 
made and the relatively minor improvements when the smaller changes are made (a 
positive change is effectively towards the observed value). The significance of this lies 
in the possibility of quantizing the correlation between prediction accuracy and the num- 
ber and/or magnitude of the non- random changes removed. 

3.11 SUMMARY OF RESULTS 

The following items were fully expected: 

a. Removal of non-random changes from the raw data has a very significant 
effect on predictions. 

b. Prediction quality is inversely proportional to the length of the prediction 
range. 

c. Prediction quality is inversely proportional to the number of non- random 
change removals. 

The following represent some surprises: 

a. The model providing the best fit to a given set of data very rarely yields the 
best predictions. 

b. All four models, in many cases, yield long range predictions varying from 
one another by only a few percent. The envelope containing the predictions 
contracts as the prediction range decreases. 

c. The removal of gross errors or outliers seems to improve the prediction 
quality. 

d. For a prediction range of 12-13 months, a value can be predicted within a 
few percent if, 

(1) No significant (>3%) data normalizations are made within that 12-13 
months, or 

(2) The total of normalization changes is relatively small in number (<4) 
and in magnitude. 
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Eitira 


Number 

of 

Points 

Linear 

Exponential 

Logistic 

Fourier 

Non-Random 

Changes 

(Percent) 

8 

68.0 

71.6 

71.9 

78.3 


9 

69.6 

73.0 

73.0 

75.0 


10 

70.6 

73.2 

73.3 

74.8 


11 

70.1 

72.6 

72.7 

69.3 

0.602 

12 

70.4 

72.0 

72.0 

70.5 


13 

71.5 

73.5 

73.4 

76.2 


14 

83.0 

84.0 

84.0 

87.0 

14.0 

15 

85.0 

84.1 

84.3 

89.1 


16 

88.4 

88.2 

87.9 

95.9 


17 

90.8 

90.8 

90.0 

96.1 


18 

92.3 

92.3 

91.3 

95.1 


19 

93.3 

93.1 

92.0 

94.6 


20 

97.0 

97.0 

94.5 

105.0 


21 

101.5 

101.5 

100.0 



22 

100.0 

100.0 

99.3 

95.3 

0.146 

23 

99.5 

99.3 

98.8 

95.3 


24 

99.0 

98.8 

98.1 

95.8 

-0.105 

25 

98.0 

98.0 

98.0 

95.7 


26 

97.8 

97.0 

97.8 

94.8 

0.063 

27 

96.9 

96.3 

96.9 

93.8 


28 

96.9 

96.2 

96.9 

96.5 


29 

96.0 

96.0 

95.8 

99.3 

-2.03 










CHAPTER 4 


COMPUTATIONAL SYSTEM DESCRIPTION 

4.1 INTRODUCTION 

The mathematical analyses of FAME can involve many numerical calculations and fre- 
quently process large masses of data. Accuracy and speed considerations dictate the 
use of a digital computer to perform these analyses. Since there are many interde- 
pendent calculations to be performed it has proven expedient to develop them as sep- 
arate programs and then use an executive routine or controller to tie them together 
into a computing "system. " The use of a system reduces the need to input data into 
each program separately and also reduces the attendant chances for numerical and 
keypunch errors. It also reduces the over-all elapsed job time since the calculations 
for several programs can be done at one "pass" on the computer or in desired com- 
binations, at the discretion of the analyst. 

4.2 COMPUTATIONAL SYSTEM 
4.2.1 BASIC REQUIREMENTS 

Two basic competing requirements are recognized in designing the computational 
system : 

a. Since large amounts of data have to be processed the system must be 
efficient to keep computer time at a practical level. 

b. The system must be as flexible as possible; FAME calculations embrace 
a wide range of operations with new techniques continually being developed 
and new outputs required. In addition, the system profits from the capa- 
bility to serve in many fields, as yet uninvestigated, to improve and re- 
fine program control in these areas. This system, as applicable to 
weight/performance control is shown in Figure 4-1. It can be thought 

of as an executive routine controlling the following individual programs : 

(1) Weight data file update program 

(2) Trend programs, including: 

(a) Maximum likelihood linear . 

(b) Maximum likelihood non-linear . 

(c) Adaptive (Fourier) exponential. 

(d) Asymptotic (logistic) exponential. 
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(3) Automatic mode selector including repeating mode analysis. 

(4) Summing program including calculation of probable error. 

(5) Spacecraft performance calculations . 

(a) Block I spacecraft. 

(b) Block II spacecraft . 

(6) Launch vehicle performance calculations for: 

(a) Saturn IB launch vehicle. 

(b) Saturn V launch vehicle . 

(7) Criticality program . 

(8) Cost program. 

(9) Output program including history plots. 

The system is not limited to the above programs. Since the design of the system is 
modular in concept, programs can be substituted or added without affecting any other 
section. In addition, any number of programmers can work on the various modules 
or programs at one time since the basic interface logic between modules is not a part 
of the individual calculation programs, but is controlled by the executive routine. This 
modular construction greatly adds to system flexibility. 

4.2.2 EXECUTIVE ROUTINE 

The executive routine can be considered in two sections. The first section controls an 
administrative program designated as "Subsystem Processor for the Apollo Computing 
Effort" (SPACE) (Appendix E - Book II of this manual). This administrative program 
is centered around a collection of seven versatile input/output subroutines. It takes 
care of file positioning, loading individual programs into core and other ’bookkeeping" 
chores required for the successful operation of the system. The large amount of data 
processed and required to be available for other programs makes this a sizeable task. 
In addition the number and size of the individual calculation programs preclude their 
being loaded into core all at once. Instead these programs are stored on peripheral 
memory (tape on drum for example) in groups or 'links. " These links are loaded into 
core by SPACE, as required. In most cases a link is composed of more than one cal- 
culation program. The judicious selection of programs for each link, to eliminate the 
unnecessary loading of programs into core, is one of the methods by which the require- 
ment of computational efficiency is met. 
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4. 2. 2.1 Input Information 

There are two sets of information which must be supplied to the system. First of all, 
the type of job must be specified. The number and type of allowable prediction tech- 
niques must be specified, and the sequence and extent of processing must also be indi- 
cated. This data allows the system to set up the required sequence of operations for 
proper execution. The second set of information is the numerical data itself. (The 
problems of processing numerical data are considered in the Appendix E - Book II of 
this manual.) 

The second part of the executive routine is the control program. The purpose of the 
control data processing operations is to provide the monitor with sufficient data to 
enable it to properly execute a job. 

To do this it must be able to interpret the control data cards which specify job type, 
extent, techniques to be used, and so forth. Certain parameters must be available 
before any computations can be made, whereas other data may not be utilized until 
much later in the job. Specifically, the control data answers the following basic 
questions . 

a. What operations are to be performed and in what sequence ? 

b . What techniques are available and what is the relative priority ? 

c . Which systems are to be analyzed ? 

d. Where is the reference data located? 

e. Are any special values required? 

f. What action should be taken in the event of an error during the job? 

g. What type of output is required ? 

4.2.3 OPERATION 

4.2.3. 1 Operation Sequence 

One of the significant features of the system is its ability to handle in sequence many 
diverse operations. Every operation, from updating the weight data file where func- 
tional system weight history data is stored to plotting final graphs, could be a part of 
a single operating system. Complete automation of any technique requires thorough 
understanding of all the factors influencing weight trends and associated phenomena. 
Additional elements can be added after exploration in greater depth to improve the capa- 
bility of the system for making the "engineering judgments" required for application of 
Forecast Analysis to a specific task. 


4-4 


In using the system, a list of the jobs to be accomplished is required. Such a list 
might consist of the following: 

a. Update the weight data file and list it. 

b. Trend the subsystem data of a given launch vehicle or spacecraft. 

c. Accumulate the total weight. 

d. Print out the results and plot the history curves, trend lines and con- 
fidence limits. 

e. Calculate deficiencies and required buyoffs for the spacecraft or launch 
vehicle. 

f. Determine criticalities associated with the spacecraft or launch vehicle. 

g. Compute costs required to accomplish the given buyoffs. 

h. Summarize and format the above data for print/out. 

The above list summarizes the important steps in Forecast Analysis for Apollo weight 
performance control. In order to convey this information to the system, a list of key- 
words is employed. The actual data card might look as follows: 

PROCESS FAME UPDATE TREND SUM OUTPT1 COMPUTE CRITICALITY 
COST REPORT* 

The first word (PROCSS) above indicates that a process or procedure is de- 
fined. The second word is the name associated with the procedure - in this 
case, FAME. The words are interpreted by the monitor and input processor 
in the same manner as the list presented first would be interpreted by an 
experienced engineer. 

An important capability of the monitor is its ability to remember a procedure. 
Once primed with the above data, the system will execute the FAME pro- 
cedure by merely specifying 

RUN FAME * 

Modifications to a standard procedure can also be accommodated without completely 
redefining the procedure . 

4.2. 3.1 Specification of Techniques 

It may be advisable for reasons of efficiency or economy to limit the number of tech- 
niques available to the system. For example, it might be deemed advisable to omit for 
a given run the Fourier model from the repertoire of the automatic mode selector. 
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In another case, it may be advantageous to specify priorities for a set of otherwise 
equal techniques. In either case, the information is used by the monitor to determine 
the programs and subroutines to be used for the run. A diagram of the over-all con- 
trol logic used by the input processor is shown in Figure 4-1. 

4. 2. 3. 2 Determination of System to be Analyzed 

The structure to be analyzed can be specified in a variety of ways . The most obvious 
method is to indicate starting and ending case numbers. Another method would be to 
merely specify the spacecraft or launch vehicle and let the program decide what func- 
tional subsystems are included. This sophistication however, does not appear to offer 
any significant advantages. 

Functional systems are identified by a seven-digit code number. The significance of 
each digit or group of digits is shown in Figure 4-2. 


X X XX x xx 

* til 


Major Class 
of S/C or L/V 


L Functional System 
Code Number 


Stage or Module 


Time Phase in Mission 


Figure 4-2. Functional System Code Numbering 


4. 2. 3. 3 Location of Reference Data 

One of the drawbacks in any large data processing program is the tremendous amount 
of data that must be made available to the system. At this point in the system design, 
the following separate tapes are required. 

a. Weight data file . 

b. Analytical data file (contains the results of the current job). 

c. Old analytical data file (referred to by automatic mode selector). 

d. Dictionary (contains standard processes and file references). 

e. Library (contains all the programs of the system). 
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This part of the data problem is very well defined. Two header cards containing tape 
data will be required for every run. The first card is needed to establish the relation 
between physical reels and the tape drives. The second card is needed to provide the 
relation between the physical reels and the program parameters. These two cards are 
the first two in every job deck. Presently weight and performance data are extracted 
from monthly reports and entered on the weight data file via input cards. Provisions 
can also be included for updating the weight data file directly from other computer 
tapes, should these tapes become available. 

4. 2. 3. 4 Data Modification 

An irritating feature of most programs is the difficulty of modifying some of the param- 
eter values without revising the programs. This problem will be at least alleviated, if 
not eliminated, in the FAME system. Any variable in core can be modified using 
header cards. 

4. 2. 3. 5 Error Control 

In a program of the size and complexity of this system, it is important to detect errors 
as soon as possible. Errors can be several types. Some can be anticipated by the 
programmer and some cannot. It is important that an effort be initiated immediately 
to analyze possible error sources so they can be monitored. 

The modular nature of the program should make it easy to handle error conditions. 

Until error analysis procedures have been established, the job will be terminated as 
soon as an error is detected. Since all data of consequence is saved on the analytical 
data file, restarting the program at the last good data should be a straightforward 
procedure . 

4. 2. 3. 6 Output Specification 

Traditionally , the rule of thumb regarding output frequency and quantity has been 
"When in doubt Print it Out. " 

This is acceptable for a small program with no permanent tape storage capability but 
neither of these two characteristics apply to this system. The quantity of output that 
can be generated by this system is overwhelming. It seems inadvisable to spew out 
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such quantities of output until quality has been analyzed. The following procedure 
utilizing three output modes is recommended: 

a. In the absence of specific instructions to the contrary, output will con- 
sist solely of FAME data. 

b. A second mode will consist of FAME data, specified checking data and 
execution comments. This mode will be specified automatically by the 
monitor if an error occurs . 

c. The last mode is the DUMP mode. Extreme care should be exercised 
in specifying this mode since the entire contents of the analytical data 
file is printed out. 

4.3 AUTOMATED RESULTS PRESENTATION 

The Forecasting data resulting from the analysis by the computer models of weight 
and performance information is received in the form of tabulation and a supplementary 
digital computer plot. Options are also available for Calcomp plots of the data. The 
three types of output available are shown in Figure 4-3. All trend programs utilize 
the same output formats. 

4.3.1 TABULATION OF DATA 

The tabulated data resulting from a typical computer run in the Forecast Analysis sys- 
tem is shown in Figure 4-4. It should be noted that only data at the functional system 
level is utilized by the computer models in Forecast Analysis. Zone 1 lists informa- 
tion applicable to the particular plot in question, e.g. , the type of trend run, the case 
title, and the case number. Also listed in Zone 1 is the data the case was run, and the 
date and number of the data file used to supply the weight and performance information 
for the run. Zone 2 lists the titles of the data columns tabulated on the form. Zone 3 
is the tabulation of weight and performance data in the observed range. Zone 4 is the 
tabulation of weight and performance data in the Forecast range. Zone 5 is the final 
forecast values for the particular case shown. 

4.3.2 DIGITAL COMPUTER PLOTS 

Plots are printed out in two forms. The first of these forms (Figure 4-3) illustrates 
the functional system plot corresponding to the tabulation of data discussed in the pre- 
ceding paragraphs. The second form (Figure 4-5) illustrates the history plot used for 
stage and module data plotting and for plotting of launch vehicle and spacecraft total 
weight. 
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Figure 4-3. Automated Results Presentation 
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Figure 4-4. Tabular Results Presentation 
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Functional system plots, Zone 1 of Figure 4-5 corresponds directly to Zone 1 of 
Figure 4-4. Since this figure is a plot of the data tabulated in Figure 4-4, use of the 
same identification number serves as an aid to clarity. The numbers on the figure 
relate to the following: 

1 . Plot of the observed weight data . 

2. The non-random changes incorporated into the data will not be seen on 
the plots. 

3. The normalized data is not a portion of the digital computer plot, but is 
shown here to illustrate how their values influence the trend line . 

4. Plot of the mean trend line in the observed range. 

5 . Plot of the mean trend line in the forecast range . 

6. Plot of the upper 95 percent confidence limit in the forecast range. 

7. The normalized data in the forecast range is plotted from the last ob- 
served point, using the mean trend line slope. 

8. No non-random changes are shown on the tabulation of results. To show 
their effect on the plots an arbitrary value has been included. 

9. Forecast line showing effect of the non- random change in the forecast 
range . 

10. The average weight growth, like the non- random changes in the observed 
range, is not a portion of the digital computer plot. 

11. The vertical scale, elected by the computer on functional system plots, 
is the weight of the system in pounds. 

12. The horizontal scale, which is dependent on a specified plot size, is the 
time in months. 

CALCOMP PLOTS 

The Calcomp graphical plotter produces output, (Figure 4-6) which has much better 
resolution than the digital plotter. This better resolution on plots is highly desirable. 
The Calcomp plotter is also capable of producing all the desired information. Symbols 
are utilized to represent different curves and these symbols are connected by straight lines , 
dashed lines , or dotted lines . Schedules and Key Events are simulated on the Calcomp 
plotter by overlapping symbols . Any information to be printed on the surface of the plot can 
be plotted, along with the points and lines . Usingthe Calcomp plotter to produce functional 
system plots will yield a sharp , clear graphical plot of the output data . However , one draw- 
back is apparent . The time to produce these plots i s excessive when compared to the time it 
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Time in Months 

Figure 4-6 . Calcomp Graphical Plotter Output 


takes to produce a digital plot. The digital plots are produced at high speed, but the 
Calcomp plotter. The actual time it takes to plot each graph is a function of the amount 
of information to be plotted, roughly 5 minutes each. Output formats for launch vehi- 
cle reliability, costs, and utilization areas for appraisals are covered in later chapters 
of this book. 
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CHAPTER 5 


REPORTING TO MANAGEMENT 


5.1 PURPOSE 

The whole purpose of FAME, as stated in Chapter 1, is to provide meaningful infor- 
mation of management planning and control. To be useful, the information transmitted 
to management must meet certain predetermined requirements. The starting point is, 
of course, to have a clear definition of what information is wanted. This seem obvious 
enough until it is recalled that many organizations generate information which is not 
useful, needed, or wanted. 

Assuming a clear-cut need for certain types of information, several questions need to 
be resolved. For simplicity, these questions can be stated as: What? When? How? 
Who? Answers to these questions may not be simple. Taking them in order, the first 
question is : What kind of information is needed or wanted ? The answer will come out 
of the nature of the subject being controlled and the depth of information wanted. In 
some cases, there may be a requirement not only for information about current status 
and for conditional forecasts, but also about possible remedial actions and their con- 
sequences, such as the time and money involved in each alternative. This brief elab- 
oration will serve to illustrate that the answer to what kind of information is wanted 
deserves careful definition. Not only does the final decision on this question influence 
the type of data flow to be established it influences the selection of math models and 
their utilization. 

The matter of when information is needed usually is resolved by the nature of the pro- 
gram being controlled. Feedback may be needed every day, every month, every 
quarter — at any interval, depending upon how soon adjustments must be made to pre- 
vent a sequence of compounded error. Since Forecast Analysis has the capability for 
looking ahead to see what is likely to happen unless corrective action is taken, the re- 
porting interval should be on a timely basis in order to capitalize on the intrinsic value 
of the system. 


The question of how the information yielded by FAME is to be transmitted refers to 
the form to be employed rather than the channel to be used. The form requirements 
are that the information be clear, concise, complete and undistorted. Clarity is 
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obtained usually by employing on a continuing basis those graphic devices which con- 
vey meaning quickly. They can employ words, numbers, pictures, symbols, lines, 
bars, etc., arranged into charts, tables, pictograms, and the like. Usually informa- 
tion will be presented on a comparative basis; for instance, with the last reporting 
period and in relationship to ultimate goal. And the base chosen should be such that 
variations and disturbances are not smoothed out, magnified, or distorted, either 
deliberately or unintentionally. 

The question of who is to receive the information is not a concern about protocol but 
about the level of refinement and the depth of detail needed in the reports. If, for ex- 
ample, the report is for top management only, the inclusion of details of value only to 
department managers merely introduces "noise" into the communication system. Very 
often, the question of who is to be on the distribution list is related to the time factor. 
If early action is a requirement in the program, a wide distribution to various levels 
can expedite program adjustment or adaptation. Sometimes this matter is resolved 
by providing monthly reports to one distribution list and quarterly reports to another, 
again depending upon the nature of the programs . 

Many of the questions which may arise about the reporting phase of FAME can be 
answered by returning to a definition of the basic purposes of this system, which are: 
to provide management with a means for determining whether a program is progress- 
ing in accordance with established goals; to supply timely predictions of possible 
future developments based upon current trends, and to furnish estimates of the proba- 
ble consequences of alternative actions. It is worth mentioning that one additional 
yield could be evidence of program planning errors or inadequacies. 

The information which can be transmitted to management by the FAME system is as 
follows (both for the over-all project and major individual elements): 

a. Current Status Review. 

b . Predicted Status . 

c . Problem Areas . 

d. Criticalities. 

e. Buyoffs (together with estimated costs in dollars and anticipated schedule 
slippage). 

This information can be graphically displayed in easily read charts. Figure 5-1 
illustrates an idealized representation of a program which progressed according to 
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plan. The abscissa is usually time, the ordinate can be weight , power, dollars, etc. 
• The curve shown is the one which best fits the plot of data points (indicated by x's) 
versus time . 



Figure 5-1. Program Status Presentation Idealized Program 


Figure 5-2 represents an actual progress report at some stage in a program. The 
solid line represents the reported values plotted versus time. The dotted line is the 
trend line, and in this illustration it predicts that the parameter will exceed its con- 
trol limit before the target date . Charts of this type are used to transmit FAME in- 
formation to management. 



Figure 5-2. Program Status Presentation Representative Program 


An additional report to management, shown in Figure 5-3, pinpoints specific problems 
and indicates the degree of criticality. In this illustration current and predicted ex- 
penditures are shown in relation to budgeted amounts. 
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Expenditures versus Budget 

Required 
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Figure 5-3. One Method for Indicating Criticality by Use of Shaded Areas 

In many cases established control limits are based upon capabilities which may change 
during the course of the program. Figure 5-4 shows two plots of reported values over 
some time period. Line A, which, let us say, is power source capability shows prog- 
ress related to the Control Limit, which in this illustration represents minimum ac- 
ceptable capability. Line B, representing total power load, rises throughout the pro- 
gram, perhaps necessitating an increased control limit, but in any case diminishing 
the criticality of a possible increased demand for electrical power relative to fore- 
casted capability. 



Figure 5-4. Graphic Illustrations for Program Visibility 

The graphic devices used to provide visibility for program managers can very in ac- 
cordance with the particular needs of a specific program. The essential requirement 
is, of course, to utilize graphics which are easily read, accurate, and timely. The 
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remainder of this section describes how these requirements are met for weight/per- 
formance control of a major aerospace program. 

For Apollo weight/performance control, the report to management is a monthly publi- 
cation entitled Apollo Space Vehicle Weight/Performance Forecast and Analysis 
Status Transmittal Report (U) (FAST), internally referred to as "the PAM." This pub- 
lication presents weight/performance comparisons down to the stage and module level. 
Once every three months current and forecast status are presented down to the func- 
tional system level. Summary charts are used to present for each space vehicle a 
composite picture of current weight/performance relationships, forecasts relationships, 
required buyoffs , criticalities, and possible trade-offs with cost, schedules, and 
reliability . 

To assure maximum utilization, the FAST is divided into three volumes which are 
separately routed to top management, system managers, and section chiefs. Needed 
information is presented in charts, graphs, and tabulations which not only point up 
current and potential problems but indicate the degree of criticality involved. A de- 
tailed description of the FAST follows. 

5-2 FORECASTS AND APPRAISALS FOR MANAGEMENT EVALUATION 
MEMORANDA 

The results obtained from Forecast Analysis can be presented in many ways. For the 
Apollo weight/performance data, the form used in the Forecast and Appraisals for 
Management Evaluation Memoranda (Acronym FAST). These memoranda graphically 
present to Apollo program management the predicted values obtained each month from 
data reported by contractors and the NASA Centers. The monthly issue of FAST pre- 
sents the current status and predicted values for weight/ performance parameters to 
the stage and module level. Once every three months, current and predicted data are 
presented down to the functional system level. Summary charts present a composite 
picture of today's weight/performance status, predicted weight/performance values, 
and required buyoffs. The summary charts are supplemented by weight/performance 
deficiency matrices and trade-off summaries for cost, reliability, and schedules. 

Each space vehicle is described on a status chart. This chart presents first-order 
mission objectives, engine performance status, summarized weight prediction data, 
stage and module trade-off factors, and primary weight/performance highlights and 
problems . 
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To assure maximum usage at all levels of management, Forecasts and Appraisals for 
Management Evaluation (U), (FAST) is divided into three volumes. Volume I contains 
summaries for use by top-level management. Volumes II and IE contain launch vehi- 
cle and spacecraft weight/performance data respectively in summary and detail form. 
The latter two volumes are directed primarily to system managers and section chiefs . 

5.2.1 VOLUME I - SUMMARY 

This volume summarizes in concise fashion the weight/performance status of the 
Apollo/ Saturn space vehicles up to and including the current month. Salient facts are 
highlighted in summaries and in deficiency and trade-off matrices. Background infor- 
mation is also provided on a vehicle-by-vehicle basis by means of status charts. 

5. 2. 1.1 Weight/ Performance Summary Chart 

The first data shown in Volume I is the Summary Chart, shown in Figure 5-5. (All 
numbers shown in figures in this chapter are fictitious.) The summary chart presents 
current, control, and predicted values of spacecraft weight and launch vehicle payload 
capability. This chart pinpoints weight/performance trouble spots at the stage and 
module level, then at the launch vehicle and spacecraft level, and finally at the total 
space vehicle level. Such a presentation allows management to focus attention on 
major problems only. The shaded patterns indicate the probable impact of critical 
vehicle deficiencies on specific missions and on over-all program objectives. Special 
situations not covered by the established form of the charts are emphasized by adding 
explanatory notes where needed. 

5. 2. 1.2 Trade-Off Summary Chart 

Volume I of the PAM also includes trade-off summary charts, represented in Fig- 
ure 5-6 . All values on this chart are predicted values at the designated shipping date. 
A weight/performance deficiency is shown in terms of equivalent payload pounds for 
each launch vehicle and spacecraft. The launch vehicle values represent payload capa- 
bility , while the spacecraft values represent payload weight . A deficiency with a posi- 
tive (+) value represents increased payload weight or decreased payload capability and 
so indicates an unhealthy condition. A negative (-) value indicates a weight reduction 
or a payload capability increase; both are healthy conditions. The cost shown is an 
estimate of dollars required to remove, by major hardware redesign, weight deficits 
in individual stages and modules. These cost estimates are based on stage or module 
inert weights , not on equivalent payload pounds. The probability of mission success 
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Figure 5-5 . A Typical Weight/ Performance Summary Chart for Volume I 
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Figure 5-6 . A Typical Trade-Off Summary for Volume I 



for the launch vehicle, the spacecraft, and the total space vehicle also can be shown. 
The schedule change column indicates the estimated number of months required to in- 
corporate any predicted weight buyoffs . Schedule change predictions are based on the 
level of effort currently being expended on each stage or module. The values given do 
D °t illustrate program status , but indicate the additional months required to maintain 
a stage within the control limit. The Pacing Action date shown is the long-range pro- 
curement date or "Today," if long-range procurement has already started. 

5.2. 1.3 Weight/ Performance Deficiency Summary Chart 

The summary shown in Figure 5-7 gives deficiencies down to the stage and module 
level. The predicted deficiencies for stages and modules of the launch vehicle and 
the spacecraft are provided along with current and control limit values for payload 
capability and spacecraft weight. The deficiencies for launch vehicle stages represent 
the loss of payload capability caused by weight growth or by performance changes in 
the individual stages. The spacecraft stage or module deficiencies represent weight 
growth of the total spacecraft due to weight growth or performance c han ges in the in- 
dividual stages or modules. Again, positive deficiencies indicate unhealthy situations; 
negative deficiencies indicate no corrective action is necessary. 

5. 2. 1.4 Weight/Performance Status Chart 

This chart is shown in two halves which face each other in the published memorandum. 
Both halves are prepared for each space vehicle. In Figure 5-8, first order mission 
objectives, mission requirements, and mission status are presented, along with weight 
and performance status information for the launch vehicle and the spacecraft. Brief 
statements describing primary problems with a specific space vehicle are made. Un- 
usual or outstanding circumstances are discussed briefly. The arrangement of the 
chart is such that it presents all data necessary for comparison of launch vehicle and 
spacecraft weight/performance status, on both a current and a predicted basis. 

The other half of the weight/ performance status chart is shown in Figure 5-9. This 
chart presents the various trade-off factors associated with each stage and module, 
plus a graphic portrayal of the weight growth prediction for launch vehicle capability 
and spacecraft weight. The plot on this chart illustrates the weight/performance inter- 
face between the launch vehicle and the spacecraft and tells what its status is now and 
will be later. Schedule data is shown at the top of the plot for launch vehicles and at 
the bottom for spacecraft. 
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Figure 5-7. A Typical Weight/ Performance Deficiency Summary for Volume I 
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Figure 5-8. A Typical Weight/Performance Status Chart (First Half) for Volume I 


























Figure 5-9 . A Typical Weight/Performance Status Chart ( 2 nd Half) 





















5.2.2 VOLUME H - LAUNCH VEHICLES 


This volume presents, in a concise fashion, the weight/ performance status of the 
Saturn IB and Saturn V launch vehicles. Also presented are Forecast Analysis re- 
sults for the complete launch vehicles, their individual stage and modules, and typical 
first-level functional systems. The weight/performance facts are set forth in con- 
venient prediction charts and numerical matrices. The prediction charts reflect pre- 
dicted weight values, authorized weight buyoffs, propulsion system performance 
changes, and required buyoffs. The numerical matrices include trade-off factors for 
weight/performance, cost, schedule, and reliability, as well as weight/performance 
deficiencies . 

5.2.2. 1 Weight/ Performance Summary Chart 

The weight/performance summary chart, Figure 5-10, is the first presentation of data 
in Volume H. This chart is very similar to the weight/performance summary chart 
described in paragraph 5. 2. 1.1 except that it only goes as high as the launch vehicle 
level. 


5. 2. 2. 2 Trade-Off Summary Chart 

Trade-off summary charts (Figure 5-11) are used in Volume H to present data on the 
cost, reliability, and schedule aspects of weight/performance deficiencies. This c ha rt 
is similar to the trade-off summary chart described in paragraph 5.2. 1.2, but gives 
deficiencies and trade-off data at the stage and module level and includes both weight 
deficits and performance deficits. 

5. 2. 2. 3 Weight/ Performance Deficiency Summary Chart 

In Volume H, the weight/ performance deficiency summary chart (Figure 5-12) gives 
deficiencies for launch vehicle stages and modules . The values are given in terms of 
both inert weight and equivalent payload weight. They represent the loss of payload 
capability caused by weigfrt growth . Loss of payload capability due to performance 
changes in the individual stages is also shown. 

5. 2. 2. 4 Launch Vehicle Payload Capability Plot 

The many aspects of the launch vehicle payload capability are presented in launch vehi- 
cle payload capability plots (Figure 5-13) . Each plot consists of a computer printout 
or history plot upon which is mounted a stage and module Prediction Analysis s ummar y 
form which gives a complete picture of the status and predictions for a particular launch 
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Figure 5-10. A Typical Weight/Performance Summary Chart for Volume H 
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Figure 5-11. A Typical Trade-Off Summary for Volume II 
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Figure 5-13. A Typical Launch Vehicle Payload Capability Plot for Volume II 




vehicle at the stage and module level, as well as current, control limit, and predicted 
values, growth rates and required buyoffs. Four classes of weight change are reported: 
authorized, pending, potential, and proposed. 

5. 2. 2. 5 Stage and Module Prediction Chart 

This chart (Figure 5-14) gives the results of Forecast Analysis for each stage and 
module of each launch vehicle. It consists of a computer printout (history plot) upon 
which is mounted a summary of the functional system data for the stage or module. 

For purposes of comparison weight and growth rates from the previous month are 
also shown. 

5. 2. 2. 6 Functional System Prediction Data 

The final form of data presentation in Volume II involves the functional system data. 
Once every three months additional books of Volume II are published which contain 
tabulations of data and computer printouts of every functional system involved in the 
Forecast Analysis calculations. These tabulations and printouts are illustrated in 
Figures 5-15 and 5-16. 

5.2.3 VOLUME HI - SPACECRAFT 

This volume presents the weight/performance status and the results of Forecast 
Analysis for the spacecraft. It is entirely similar in form and make-up to Volume II, 
just described in the preceding paragraphs. The only difference between the two 
volumes is that Volume II discusses or presents launch vehicle payload capability 
while Volume III presents the total spacecraft weight (at injection). Volume HE, of 
course, also presents data on spacecraft stages and modules rather than launch vehi- 
cle stages and modules. Since the two volumes are so similar, a detailed discussion 
of Volume HI is not necessary. 

5.3 INTEGRATED WEIGHT/ PERFORMANCE STATUS AND ANALYSIS 

5.3.1 GENERAL DISCUSSION OF THE "STATUS AND ANALYSIS" DOCUMENT 

The Apollo Space Vehicles Integrated Weight/ Performance Status and Analysis (U), is 
a document prepared and published in the interim between issues of the Forecast 
Analysis Memoranda. It presents only the most salient facts of the latest Apollo 
weight/performance developments. Issued each month, as soon as possible after 
receipt of current data from the individual contractors, the document is published 
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Figure 5-14. A Typical Stage Prediction Chart for Volume II 
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Figure 5-16 . A Typical Functional System Plot for Volume II 



prior to the completion of the Forecast Analysis computer runs which form the basis 
for the detailed Forecast Analysis Memorandum. 

5.3.2 PURPOSE AND PREPARATION OF THE "STATUS AND ANALYSIS" 

DOCUMENT 

The primary purpose of the "Status and Analysis" document is to give on a quick 
response basis a clear, concise, and integrated summary of the current Apollo/ Saturn 
mis sion-by- mission status, together with an approximate analysis of probable future 
trends. In the interest of time no effort is made to incorporate the newly received 
data into the prediction trends. Instead, predictions are made by taking the current 
data points and using them to project the previous month's growth rates. This method 
gives a fairly good approximation to the true prediction because the latest point gen- 
erally has a relatively minor effect on the growth rate when incorporated into a pre- 
diction already based on a large number of history points. 

5.3.3 DESCRIPTION OF THE "STATUS AND ANALYSIS" DOCUMENT 

The Apollo Space Vehicles Integrated Weight/Performance Status and Analysis (U) document 
briefly summarizes the weight/performance analysis of the complete spacecraft, and 
the over-all launch vehicle payload capability required to accomplish the specified 
mission. Details and analyses of second generation mission components are purposely 
omitted and left to the more extensive Forecast Analysis Memorandum. The "Status 
and Analysis" document consists primarily of three parts, the introductory letter, the 
trade-off summary, and the weight/performance summary. Each part is designed for 
a specific function. 

The introductory letter tells the purpose and intent of the document and refers 
to the source of a more detailed analysis, if desired. The trade-off summary 
(Figure 5-18) shows the weight/performance deficiency for the spacecraft, the launch 
vehicle and the over-all mission. It interprets the deficiencies in terms of probable 
cost, loss of reliability, and schedule changes which may be involved in their elimina- 
tion. Areas of particular interest and importance are outlined by notes under "High- 
lights and Problem Areas, " as a supplement to the trade-off summary. 

The weight performance summary (Figure 5-19) identifies the type of mission to be 
accomplished and gives the current, control limit, and forecast values for both space- 
craft weight and launch vehicle capability. The entire history and forecast lines for 
spacecraft weight and launch vehicle payload capability are plotted versus time, along 
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Figure 5-18. Typical Overall Summary Tabulation 
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Figure 5-19. Typical Mission Summary Plots 
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with their control limits. The forecast lines are extended out to the shipping dates of 
the spacecraft and launch vehicle. This graph shows not only the relation of the units 
to their control limits, but also their relation to each other. Areas are highlighted 
where future trade-offs might profitably be considered. 

5.4 OTHER TYPES OF OUTPUTS 

5.4.1 INTRODUCTION 

Publication of the Forecast Analysis Memoranda and the integrated weight/peformance 
status and analysis requires the compilation and integration of a large amount of data. 
A long working involvement with this data and component relationships of the Apollo 
missions leads to insights into facets of the weight/performance area that suggest the 
need for a special analysis of an area of interest of the generation of a report on a par- 
ticular aspect of the mission. Examples of such special applications follow. 

5.4.2 INDIVIDUAL VEHICLE/STAGE/DRY WEIGHT SUMMARIES 

Many of the inert weights used in the analysis of mission capabilities and performance 
include a certain amount of fluids in the form of residual and reserve propeUants. 
Certain types of studies, however, require a knowledge of the dry (or hardware) weights 
of individual stages and modules. For example, an analysis may be made of produc- 
tion costs in relation to total hardware weight. The Dry Weights Report (Figures 5-20 
and 5-21) fills this need by listing total hardware weights for the stages and modules of 
each mission. 

5.4.3 SUMMARY BY DESIGN RESPONSIBILITIES FOR PROGRAM CONTROL 

The Apollo Program Control Office has final responsibility for coordinating and con- 
trolling individual contractor performance on the program. Contractors are held to 
rigid weight control specifications. It is necessary that program control be madp 
aware at the earliest possible time of any deviations or of any trend that may indicate 
future deviation. Therefore, the stages and modules are summarized by design re- 
sponsibility in the Dry Weights Report. Notes are used in the FAST to serve the same 
function, particularly in those areas which seem headed for trouble. 

5.4.4 SPECIFIC AREA COMPARISON - INTERFACE CHARTS 

Reporting and trending individual components of a complex and integrated system does 
not give the whole story without a presentation of the relation between those components. 
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Figure 5-20. Spacecraft Dry Weights Summary 
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Figure 5-21. Launch Vehicle Dry Weights Summary 
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This need is met in part by interface charts. The function of the interface chart is 
to present the components of a system as integrated whole rather than as individual 
stages and modules plotted against their specified control limit as isolated cases. 

These charts display the capability of each system to perform its specified function. 

To show the importance of this interface data the following questions might be asked. 

Is the S-IB stage of the SA-504 launch vehicle capable of achieving the necessary veloc- 
ity to perform its function considering its own weight plus the weight of the stages and 
modules it must carry ? Is the LEM-De scent stage capable of performing its specified 
mission in light of the fact that the LEM-Ascent stage, which is must carry to the 
lunar surface, is predicted to exceed its control limit? Does the predicted capability 
requirement for any stage indicate a need for more propellant than its tanks will hold? 

An integrated picture is achieved by plotting the forecasted capability of each compo- 
nent against the trend of the total requirement. The point at which these lines cross 
(i.e. , the point at which the requirements exceed the capability) indicates a potential 
source of problems. These plots may also reveal possible trade-offs. For instance, 
even though a system may not meet its control limit value or may not be capable of 
meeting the capability specified for it, it is possible that the requirements are not as 
rigorous as supposed or specified. In this case, an analysis should be made to deter- 
mine if a trade-off is possible. 

5.4.5 WEIGHT DATA COMPILATION 

One of the most useful results of Forecast Analysis is the Weight Data File . This file 
is a by-product of the Forecast Analysis effort and is not issued as an official report. 
Publication of the FAST involves collecting and recording extensive amounts of data 
over extended periods of time . This data is summarized and stored in one central 
location, the Weight Data File. This file is simply a printout of the recorded values 
for each individual component of the Apollo missions. Each component is identified 
by a code number and the recorded values of that component are listed by month through- 
out the period of time for which data is available. This data is stored on computer 
tapes and is printed out each month after the latest data has been added. By this means, 
the complete history, or the specific value at any point in time, is readily available for 
any system in question. The Weight Data File makes no attempt at analysis or expla- 
nation, but simply serves as a central data coHection point and ready reference for 
recorded and reported data. 
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5.4.6 NOTES ON PROBLEM AREAS 


The presentation of data and the results of calculations in the form of tables, charts, 
and graphs does not in itself present the complete picture of the problems which may 
be involved on any Apollo mission. To tie the various components into a comprehen- 
sive picture and to present and explain specific problem areas , extensive use must be 
made of explanatory notes. These notes are another by-product of Forecast Analysis. 
They may be derived directly from contractor reports or they may be the result of 
insights gained through processing of reported data. Such notes are a useful and nec- 
essary part of the effort to present an effective and comprehensive summary of a sys- 
tem which is as complex and interrelated as the Apollo system. 
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CHAPTER 6 


APPLICATION OF FAME TO OTHER TECHNICAL AREAS 

6.1 GENERAL APPLICABILITY 

This chapter discusses the application of FAME to areas such as cost, schedule , vehi- 
cle performance, and electrical power surveiUance. It is suggested that techniques 
described in this manual may be used wherever the foUowing elements are available: 

a. Measurable requirements. 

b. Status information flow. 

c. Measurable status. 

When these three elements are present, patterns of data behavior may be measured 
and assessed. The measured status compared with measurable requirements for ms 
the basis for forecasting deficiencies which require managerial analysis and action. 

Through the Forecast Analysis computational system, which uses elements a, b, and 
c above, the decision-maker is provided with information in an effective, formal out- 
put. The Forecast Analysis computational system considers the relationships, inter- 
dependencies and interactions between the elements listed above. These relationships 
are shown pictorially in Figure 6-1. Historical values of the measured parameters 
form the basis for forecast values. They also give confidence limits for the predicted 
values. As a program matures FAME techniques provide consistent data in usable 
form for managerial action. Thus, by using a formal evaluation program, the gap is 
closed between the information specialist's capacity to understand the information for 
assessment and decision making. 

6.2 APPLICATION TO COST AND SCHEDULE MONITORING 

Launch vehicle or spacecraft weight growth is usually paralleled by increased expendi- 
tures and schedule modifications. It seems apparent that a FAME system can be de- 
vised for cost and schedule control by utilizing the same basic approach now in effect 
for weight control. FAME offers several advantages not provided by present methods 
of control. This section discusses the basic requirements for instituting a FAME sys- 
tem for cost and schedule control to obtain these advantages . 
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Figure 6-1. FAME Computational System Elements 
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6.2.1 THE IMPORTANCE OF COST AND SCHEDULE MONITORING TO 
MANAGEMENT CONTROL 

During a development program of any type , management control is particularly im- 
portant in the area of program cost and schedule. Budgets, profits, and reputations 
are aU sensitive to the ability of management to meet schedule and cost commitments. 
Money is supposedly spent at aHocated rates during the progress of a program. Simi- 
larly, schedule milestones are established to set and measure the program's pace. 

If, at some point in time, progress begins to lag in a certain area, schedules begin to 
sHp and invariably costs wiU mount. Any delay in pointing out such problem areas to 
management permits additional slip and involves greater expenditure . Once alerted, 
management can initiate corrective action immediately. But harm has already been 
done, for money once spent cannot be "unspent" — at least not without degrading 
equipment quality elsewhere . Schedule slips usually can be made up , but only if suf- 
ficient funds are available . The problem thus becomes a multi-parameter trade-off. 

6.2.2 PRESENT COST AND SCHEDULE TECHNIQUES 

For the analysis of cost and schedule performance, it is necessary to organize a 
means for obtaining necessary data. Several exceHent program management tech- 
niques are presently available which provide the necessary data. These can be roughly 
classified as control or forecasting techniques. A brief look at some of these tech- 
niques will help determine what information is or is not presently available to 
management . 

Probably the most widely accepted and successful means for controlling schedules and 
costs in PERT (Program Evaluation and Review Techniques). PERT permits accurate 
planning, indicates current progress, and provides a certain degree of advance warning 
for potential trouble areas. This technique is especiaHy appMcable to non- repetitive 
operations such as construction and development programs. However, the objective 
in most development programs is to produce a product which is competitively superior 
in three respects; cost, delivery time, and quality or reliability. PERT aids mainly 
in meeting delivery dates. It has little effect on cost and quality factors. PERT is 
an excellent tool for realistic, advanced planning of program schedules. It provides 
an adequate means of monitoring schedule progress. It can also point out potential 
weaknesses. However, a FAME system can draw attention to schedule problems at 
an earlier point in time, thus improving management control. 
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A more serious need for control exists in the area of cost. PERT does not provide an 
early indication of increasing cost trends. FAME on the other hand, offers the same 
distinct advantages gained in weight control — early problem indication, a measure of 
the severity of problems, and quick reaction capability. 

Such techniques as NASA's "Program Analyses and Evaluation Procedure (PAEP)" or 
"Launch Vehicle Cost Model" are invaluable for establishing long-range plans. Many 
of the mathematical techniques developed for them will be of value in FAME techniques. 

6.2.3 COST AND SCHEDULE MODEL FOR FORECAST ANALYSIS 

The needs of management and some of the gaps left by present systems indicate that a 
mathematical model for Forecast Analysis must be developed which can be applied to 
the control of costs and schedules. The data retrieval and processing system required 
to implement the analysis must also be considered during the development. 

6.2.3. 1 Model Objectives 

The primary purpose of the cost and schedule model is to yield information which will 
in turn provide management with a program control system that will: 

a. Evaluate the program status. 

b. Forecast values for cost and schedule factors. 

c. Detect areas of potential weakness at the earliest possible moment. 

d. Determine the seriousness of problem areas. 

To accomplish this, a number of specific items must be fulfilled. Periodic reporting 
procedures must be established to insure data for current and adequate evaluations. 

The model to be developed should be capable of monitoring the growth of expenditures 
and the schedule status as the program progresses. The model should contain the 
schedule milestones and the cost budget figures under which the program is constrained 
to operate. Based on the data reported, forecasts of the most likely future trends can 
be made. The development of forecasting procedures involves the analysis of past data 
to determine the relationships between cost or schedule factors and the pertinent 
parameters in the environment. It also involves the development of mathematical 
forecasting techniques which adequately represent these relationships. The model 
must have the capability of parametric variation. This requirement is necessary to 
provide management with sufficient data to trade-off one course of action against 
another. If desired, the model might be expanded to conduct a certain amount of trade- 
off analysis internally, thus limiting the decision making to those alternatives which 
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hold the most promise for success. The cost and schedule program structure is 
represented graphically in Figure 6-2. 

6. 2. 3. 2 Scope of Model 

The number of functions within the scope of the model is related to the responsibilities 
of the managers who will use the results obtained. All functions over which manage- 
ment has responsibility or a need for control should be included. Additional functions 
related to the primary functions should be included when they reveal significant pat- 
terns of dependency with the system. 

6. 2. 3. 3 Level of Analysis 

The level of complexity at which the model will be constructed is itself a subject for 
trade-off analysis. Models of lower-level activity provide more direct control over 
operating performance. However, conclusions drawn from such models are more 
susceptible to error. Models at higher levels generally can be developed and imple- 
mented more quickly and with less expense, but they are less specific. The model 
should be developed to provide a sufficient amount of control at the management level 
for which it is intended. 

6. 2. 3. 4 Selection of Variables 

Once the scope of the model and the level of its analysis have been decided upon, the 
variables to be included then must be defined. They may have been fairly well defined 
when the model scope was decided upon. In any case , elements which contribute to 
program cost and all scheduled milestones in the program must be included. 

The cost of a development program may be divided into several parts , each requiring 
different treatment in the forecasting procedures. The following categories should be 
included in program cost definition: 

a. Research and development costs: The hardware items involved in a 
research and development program may be classified as state-of-art 
items (to be acquired "as is"), items which require modification, and 
items which require complete development. Growth models for costs 
of the first and second type might be satisfied by linear characteristics. 
Costs of the last type might be described by the familiar exponential 
growth models. Investigations into growth characteristics of past pro- 
grams should aid in establishing appropriate models. 
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Figure 6-2 . Cost/ Schedule Program Structure 




b. Production cost : This category includes such areas as fabrication, 
assembly, system testing, and all other costs which follow after com- 
pletion of the initial design cycle. These are costs which are directly 
attributable to the hardware . 

c. Facility development cost : This category describes the cost of any ex- 
pansion, modification, or construction of facilities required to attain 
the objectives of the program. In some cases this cost may be prorated 
over a number of other programs. 

d. Support costs : All costs normally classified as "overhead" are included 
in this category. It might be considered one of marginal importance and 
included only if a specific need exists . 

It appears to be advantageous to select variables for the schedule model which are 
based upon present PERT networks. PERT networks used by a particular level of 
management usually represent the depth of analysis and the degree of control which 
are desired. A schedule status and trend forecasting model could then be based upon 
scheduled PERT events . 

6. 2. 3. 5 Establishing the Relationships 

Once the variables of the model have been settled upon, it is necessary to establish the 
relationships which exist between them. It is this step which distinguishes a cost 
model from an accountant's ledger. The increased cost of one subsystem may either 
increase or decrease the cost of some other. The relationships between subsystems 
can be arrived at only through a knowledge of other system parameters such as weight, 
schedule, and reliability. But it is these very subtleties which make forecasting 
models more accurate and valuable tools. The relationships between the selected 
variables can be established by analyzing historical data from present and past pro- 
grams. Appropriate mathematical trending models then can be selected to describe 
the growth with time. In addition to historical data on the parameter itself, several 
other potential areas exist for determining growth characteristics . Among these are 
the correlation between weight growth and schedule progress , and between expenditure 
of funds and schedule progress. Once established, the relationships serve as a new 
source of data, supplementing present sources and providing additional trade-off 
capability . 

The use of PERT networks as the basis for the schedule model can provide the nec- 
essary relationships between events in time. The effect of late or early completion of 
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one event on the over-all program completion can then readily be observed. The es- 
tablished relationships would also illustrate various alternatives. 

6. 2. 3. 6 Control Limits 

The progress of a program with respect to schedule and cost is judged by comparing 
their status with some predetermined criteria. In this case, the criteria are budgets 
and schedule milestones. These criteria are similar to the control limits placed upon 
spacecraft weight. Just as weight must be kept below some maximum value, so must 
cost and time. Any values in excess of these limits must be reduced, or increased 
limits must be provided. Therefore, the limits included in the model must be capable 
of changing with time . 

6. 2. 3. 7 Methods of Control 

The model might also be used to evaluate the various alternatives available . The 
model itself describes the interactions and dependencies present in the real system. 
Presumably, the result of any change made in the model is indicative of the expected 
result in the real system. Thus when unfavorable trends are noted, the model could 
be further exercised to determine the effect of resources reallocated in different man- 
ners. The alternatives could then be presented to management in a quantitative fashion 
for their evaluation and action. 

6. 2. 3. 8 Mathematical Techniques 

The use of mathematical techniques is required for the analysis of past data to de- 
termine suitable forms for extrapolation into the future. Two areas in particular are 
of interest: 

a. The analysis of past data to determine the mathematical form of growth 
characteristics . 

b. The analysis of past data to determine the functional relationship be- 
tween variables. 

The techniques available range from graphical methods, when few variables are pres- 
ent, to sophisticated multiple regression curve-fitting methods. These techniques are 
well documented in later chapters . Thus the techniques required to apply FAME tech- 
niques to the control of costs and schedules are available. 


6-8 



6.2.4 DATA RETRIEVAL AND PROCESSING SYSTEM 

An important factor in determining the ultimate worth of the output of an information 
system is the data retrieval and processing system on which it is based. The informa- 
tion system should be capable of detecting areas of potential weakness at an early date. 
The speed of a data retrieval and processing system must be an area of vital concern. 
The data being reported must be accurate and current. Present schedule reporting 
techniques, such as PERT, go to great extremes to insure accuracy of the reports . 
Cost reporting is a record of actual spending and is usually quite accurate. In Fore- 
cast Analysis, emphasis should be placed upon obtaining current data. 

6.2.5 COST AND SCHEDULE RESULTS 

The remaining step is to present the results to management in a form which will aid in 
the decision-making process. Total harmony between the system and the user can be 
obtained only after consultation, trial, and feedback. A lot of information is available 
from the computer program itself. The computer information may result from a num- 
ber of different methods. It may be available at various levels of system complexity. 
Therefore, it is the user who must ultimately determine the type and quantity of infor- 
mation which is of greatest benefit. Some typical results of Forecast Analysis for 
cost and schedule are shown in Figures 6-3 through 6-9. 

6.3 APPLICATION TO RELIABILITY MONITORING 

6.3.1 RELIABILITY ESTIMATES 

Reliability is defined as "the probability that a system, subsystem, component, or 
part will perform its required function under defined conditions at a designated time 
and for a specified operating period. " The reliability estimates made at any time dur- 
ing equipment development are really forecasts of the reliability at a specific future 
date . They are based on an evaluation of the current system and its components . Any 
changes in the system or its mission results in a change in reliability estimate values. 
Because reliability estimates are forecasts the estimates are based not on the methods 
of calculating reliabiHty , but rather on those phenomena which cause the reliability 
estimates to change . 
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Figure 6-3. Top Level Program Control of Over-all Spending 
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Figure 6-4. Spending Rate and Function Control 
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Figure 6-7. Milestone Completion 
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6.3.2 THE PURPOSE OF RELIABILITY PREDICTION ANALYSIS 
Current reliability programs which estimate the completed system reliability per- 
formance are based on: 

a. A topological system model. 

b. Subsystem and component current reliability estimates. 

c. Historical component reliability data. 

These measures of reliability performance are based on the current system configura- 
tion and on the current status of system components. As system development pro- 
gresses, the reliability estimates will change. These changes are due to several 

causes: 

a. Component reliabilities become better defined. 

b. Estimates show deficiencies which are corrected. 

c . The system reliability model is improved as system definition 

progresses. 

Reliability Forecast Analysis involves the application of techniques which predict the 
future value of reliability estimates. The forecasts are based on three fundamental 
considerations : 

a. Historical reliability trends of similar systems. 

b. The growth of reliability estimates from system inception to completion. 

c. The mechanisms of reliability growth. 

Reliability Forecast Analysis activities are not a duplication of current reliability pro- 
grams. Instead, they use the results of these programs to achieve their objectives. 
Reliability Forecast Analysis provides a method of extrapolating past and current 
reliability estimates to give future reliability estimates. In this way , Forecast Analy- 
sis provides a basis for anticipating future deficiencies and so provides another factor 
for management decisions. Trends and forecasts of reliability, weight, cost, and 
schedule, coupled with trade-off analysis, provide program management with a com- 
prehensive overview of system development and a comparative basis for resource 
allocation. 

6.3.3 THE PRE SENT RE LIABILITY ESTIMATION PROGRAM 

Because testing of a large system during design and development is not always possi- 
ble, the reliability of the system is determined by using a mathematical model. The 

model simulates the reliability requirements of the system in a phase-by-phase mission 


6-14 



sequence. In the Apollo program the primary model used is SOAR in (Simulation of 
Apollo Reliability) . SOAR is a system of computer programs developed to aid in the 
reliability estimation work on the Apollo Mission. SOAR combines analytical and 
Monte Carlo techniques to provide the speed and versatility necessary for detailed 
analysis of large, complex systems such as Apollo. The application of SOAR requires 
an accurate representation of the system. This representation takes the form of a 
reliability model showing the functional relationships of the equipment makin g U p the 
system. The reliability model may vary with time to reflect the variations in equip- 
ment while in operation. The model also provides a means for representing several 
characteristics of manned space flight. At present, no attempt is made to trend data 
or to otherwise forecast future estimates. 

6.3.4 USING ESTIMATION RE SULTS FOR FORE CAST ANALYSIS 

Reliability estimation for the Apollo program has been in operation for over two years . 
The mathematical models have been extended and improved to increase both efficiency 
and scope. Reliability estimates constitute the primary input data for Forecast Analy- 
sis of weight growth as it affects reliability. The past estimates of reliability could 
provide background data to develop trends of the estimates. The past estimates could 
also be used to determine reliability growth and to forecast future reliability estimates. 

The SOAR m model records the results of the trials in each element of the model for 
each phase of the mission. Therefore, information on mission reliability for each 
functional subsystem is available for use in estimate forecasting. Also recorded is 
data which can be used to compute the effect of functional system reliability on mis- 
sion reliability. 

6.3.5 RE LIABILITY GROWTH 

Methodologies for estimating reliability growth involve the following three elements : 

a. System reliability description. 

b. Reliability growth mechanisms. 

c. Mathematical techniques. 

The system reliability description is largely available for the Apollo program. How- 
ever, it will be necessary to coHect and supplement the background data for the 
description. 
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There are man y different reliability growth mechanisms . Reliability growth occurs 
when increased equipment information causes subsystem and unit reliability estimates 
to be revised upward, or when the increased knowledge causes improvements to be 
made. The increases in system knowledge which change reliability estimates directly 
are typified by a new circuit which has a higher inherent reliability (fewer parts, more 
conservative power rating, etc.) than previous circuits on which earlier estimates had 
been based. Increased system knowledge might also result from tests which indicate 
that original estimates of reliability had been too conservative. 

Another source of reliability estimate improvement is the indirect effects of system 
knowledge. Increases in system knowledge which produce improvements may evolve 
from criticality analyses which show that certain units are principal contributors to 
unreliability and so result in upgraded modifications . Another possibility which leads 
to corrections include unit and subsystem testing. Some other factors which lead to 
reliability growth are : 

Removal of defects . 

Insertion of redundancies. 

State-of-the-art advances. 

Redesign of critical elements . 

Substitution of more reliable devices. 

Upward revision of estimates. 

As a program progresses through the development cycle, reliability continues to grow. 
Reliability estimates do not stop growing until efforts to improve reliability stop. The 
prediction of reliability growth, therefore, must be based on a situation which changes 
during the development cycle. Figure 6-10 shows a curve shape which might be ex- 
pected for a plot of reliability versus time. During the early stages of development 
the estimates will tend to be low as in Zone A. As the equipment definition becomes 
more complete and deficiencies are corrected, the curve will rise more sharply over 
a time period. This is shown in Zone B. Then, as the system matures and reliability 
goals are approached or met and emphasis on reliability improvement decreases, the 
curve will level off, as in Zone C. At the end of the cycle, the system configuration 
is firm and all units approach the state-of-the-art in a reliability sense . 
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Figure 6-10. Reliability Estimate Growth 

Much of the literature on reliability estimation describes reliability growth from the 
viewpoint of two types of failure, inherent unreliability, and transient unreliability. 
These are defined as follows: 

a. Inherent unreliability - that portion of unreliability which cannot be elimi- 
nated by corrective engineering action due to the state-of-the-art or 
over-all design philosophy limitations. 

b. Transient unreliability — that portion of initial unreliability amenable 
to modification or elimination by en gin eer ing change. 

The inherent unreliabilities establish an upper bound on system growth. Transient 
reliabilities are detected and corrected at a rate proportional to their number. As the 
equipment is tested its reliability grows exponentially. Its reliability therefore ap- 
proaches the upper bound asymptotically - While this approach intuitively seems good 
for reliability growth, we are interested in the growth of reliability estimates . The 
principle difference is that reliability growth is based on the removal of weaknesses 
as detected, while reliability estimate growth results from the reduction in the effects 
of known weaknesses. Therefore, we must consider also the nature of the estimates 
which are being used . 

In Forecast Analysis techniques used for weight forecasting, estimates are modified 
according to the percentages of the estimate attributable to Estimated, Calculated, and 
Actual (E/ C/ A) weights. This technique is also applicable to reliability est imat e 
growth. Certainly, the estimates made before and after fabrication and testing of 
equipment can be expected to differ. Even if improvements are not made a certain 
increase in confidence should lessen the conservatism of the estimate. Corrections 


6-17 



made during test will result in improved test results and hence improve estimates. 

Thus, the refinement or growth potential of any estimate is a function of the point in 
the development cycle at which the estimate is made. It follows that a weighting 
process, akin to E/C/A technique should be used in forecasting reliability estimate 
growth. 

This manual discusses math models used in weight growth forecasting. A math model 
of particular interest here is the exponential model, of the form 

, -ct 

y - a - be 

The observed value w . is assumed to have the form 
w . = y. + e. 

—l J i “i 

where w. is composed of three parts, Estimated, Calculated, and Actual. Then the 
derivation continues to produce maximum likelihood estimators for a, b, and c based 
on measurements to date. 

This estimation model can be adapted to the reliability estimate growth forecast by 
replacing the E/C/A with an N part representation of system maturity. For example, 
take N = 2 and let D = percent of estimate based on design and H = percent of estimate 
based on hardware. This seems to be a logical starting point. The use of these 
weighting functions should permit curves having the general shape shown in Figure 6-3 
to be fitted to past estimates. Once the curve is fitted and the parameters of the model 
determined, forecasts can be made. The model above may not be the best technique 
available. Other methods and models should be considered for applicability. 

6.3.6 IMPLEMENTATION 

Reliability estimate information can be obtained from the current Apollo Reliability 
and Quality Assurance Program. This information is contained in either the quarterly 
status reports or is available from the SOAR in output data files. The status of the 
equipment at the time past estimates were made, i.e. , the values for D and H is 
available for reliability estimation purposes. Other useful information is available 
also such as from MPC250-1 and from the R&QA quarterly reports. Obtaining mean- 
ingful estimates of D and H (or whatever N part description is used) for forecasting 
purposes appears to be feasible. The quarterly frequency of reliability estimation 
reports, precludes monthly updating of reliability estimation forecasts. A quarterly 
cycle would be required. The estimate forecasting output would yield management 
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information having the same general character as that in the current Apollo weight/ 
performance reports. The reliability information would include: 

a. Current reliability estimates for the functional subsystems over com- 
plete mission. 

b. Current reliability estimates of mission success. 

c. Forecast reliability estimate growth curve for functional subsystems. 

d. Forecast reliability estimate growth curve for total mission success. 

6.4 APPLICATION TO GROUND FACILITIES UTILIZATION 

FAME is applicable where time-varying status is to be compared a gain st preset re- 
quirements. There are numerous potential applications of FAME to the utilization of 
ground facilities. Typical applications include: 

a. Utilization of test facilities and equipment. 

b. Real time transmission and delayed transmission of data. 

c. Processing of data, including monitoring. 

d. Incorporating Forecast Analysis techniques directly within the checkout 
equipment for improved performance and reliability enhancement. 

Two of these areas, c and d, are discussed below. 

6.4.1 DATA PROCESSING 

Because of the mounting complexity of test measurements and instrumentation, auto- 
mated checkout and acceptance test equipment has been developed for the Apollo pro- 
gram (see Figure 6-11). This computerized equipment facilitates rapid and accurate 
testing of flight equipment. The resulting increase in the amount and rate of dataflow, 
however, has been substantial. Surveillance of total data processing appears to be 
"a must" for the successful management of test data and for pinpointing problem areas 
revealed by the test. Surveillance would seem to be valuable in both input and output 
data handling requirements now that the number and rate of input sensory stations is 
increasing and huge amounts of output data have to be processed and stored. For 
these reasons it appears worthwhile to consider Forecast Analysis techniques for test 
data processing, particularly during the planning stages when data processing can be 
factored into program management. 

6.4.2 CHECKOUT EQUIPMENT 

Improving testing techniques and enhancing their reliability is another possible appli- 
cation of Forecast Analysis. Checkout equipment could presumably use appropriately 
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Figure 6-11. Typical ACE-S/C Control Room 







* modified Forecast Analysis techniques to sense trends in aerospace equipment per- 
formance. This appears to have value not only for acceptance testing, but also in 
determining the expected life or the time required for a critical measurement to ex- 
ceed its control limits. In many components measurements of performance can be 
made more accurately and precisely than required by operating tolerances. Using 
such measurements, trends within the control bounds can be sensed and assessed by 
Forecast Analysis. These trend appraisals can be used to select equipment with very 
narrow operating limits. This would have the same saluatory effect on reliability as 
though the control limit band had been narrowed. In addition, better predictions about 
when failure will occur can be made simply by tracking the actual performance param- 
eters and applying standard Forecast Analysis techniques. The improved reli abili ty 
data comes, potentially, from the improved understanding of the behavior of a specific 
element measured against its requirements. Pre-flight performance assessment 
might also be enhanced by improved prediction of performance at the time of shipping. 
Both a performance prediction and an associated probable error could be obtained by 
Forecast Analysis derived from data on the behavior of systems during ground check- 
out and acceptance testing. 

6-5 APPLICATION TO PERFORMANCE ASSESSMENT 

The application of FAME to we ight/ pe rfo rmanc e data is primarily based on the assess- 
ment of weight. Performance effects are included only by the use of control weights 
which are derived from performance assessments. This has merit in that the princi- 
pal effects of weight changes can be studied on a constant performance basis, reflected 
by constant control limits. Buyoffs for launch vehicle payload capability and space- 
craft total weight are calculated using the control limits, predicted spacecraft weight 
and predicted launch vehicle performance. When comparing total vehicle buyoffs with 
the sum of stage or module buyoffs some inconsistencies appear, but they are usually 
small. Thus current techniques appear to be capable of monitoring both weight and 
performance, but only when performance is included as a constraint on weight. 

Weight/performance control techniques can be improved by more thorough surveillance 
and interpretation of performance parameters . There are several ways this surveil- 
lance and improved prediction analysis of performance might be achieved. These 
include : 

a. Compare the expected performance with requirements at the stage or module 
level (for which control weights are established). That is, can the vehicle 
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perform the designated mission using both current and predicted perform- 
ance factors and assuming the weights to be equal to the control limits. 

b. Monitor performance values and compare them to their specific control 
limits, e.g. , compare current and predicted Isp to the control Isp values 
which are used for mission determination. 

c . Assess the current and predicted over-all mission performance versus re- 
quirements by a comprehensive trajectory calculation. This would require 
two long computer runs for each comparison, using current values and pre- 
dicted values, and a third run for reference using control values. 

d. Approximate c (above) using performance trade-off factors which represent 
combinations of performance effects on the over-all mission. New factors 
would be obtained each month for current and predicted status by differential 
methods discussed in Appendix D. These factors would be compared to the 
required performance factors. 

Method (b) will be used to illustrate the application of FAME. Methods (a) and (c) 
require rather extensive computational capabilities, including the analysis of complex 
trajectory phenomena, and will not be considered. Method (d) has merit, and is de- 
scribed, with illustrations of its use, in Appendix D. 

Method (b) presupposes that management, supported by systems engineering groups of 
the various design agencies, can and does establish control values for the many factors 
that affect performance. Current status is assumed to be reported on a periodic basis. 
The FAME formula could then be applied to produce quantitative information about the 
performance values. 

A list of typical performance elements which might be monitored is presented in 
Table 6-1. Such a list can be constructed for any aerospace program. This list 
includes: 

a. Items which have a measurable impact on performance. 

b. Items which have preset control limits. 

c. Items whose status can be measured and whose future value can be pre- 
dicted. 

This list shows that there are a large number of performance factors, all of which are 
important to mission success, which could be monitored. The selected factors must 
be monitored for each specific vehicle and each specific mission to present the over-all 
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Table 6-1 

Typical Performance Elements Which Could be Assessed by Forecast Analysis 


A. SPACE VEHICLE PERFORMANCE FACTORS 

1. Time Sequencing Performance 

1. 1 Hold down timing 

1. 2 LES jettison time 

1.3 Time in orbit 

1.4 Boost between stages 

1. 5 Trip time to moon 

1. 6 Stay time on moon 

1. 7 Trip time to earth 

2. Thrust or Total Impulse Tolerance 

2. 1 Stage thrust profile 

2. 2 Stage nominal thrust 

2. 3 Programmed mixture ratio profile 

2.4 Variable thrust engine performance 

2. 5 Reaction control unit thrust program 

3. Specific Impulse Values 

3.1 Engine instantaneous I S p 

3. 2 Stage average Ig p 

3. 3 Programmed mixture ratio profile 

3.4 Variation of I S p with run time, altitude 

3. 5 Variation of I sp with trottled engines 

4. Propellant Loading Tolerance 

4. 1 Stage or module loading 

4. 2 Roiloff and other losses 

5. Propellant Residual Variations 

5. 1 Stage or module residule propellant 

5. 2 Stage or module pressurization gas 
weight variation 

B. TRAJECTORY PERFORMANCE FACTORS 

1. Launch Parameters 

1. 1 Date 

1.2 Time 

1.3 Azimuth 

2. Parking Orbit Parameters 

2. 1 Altitude 

2.2 Ellipticity (Ephermeris) 

2.3 Epoch 

3- Injection Parameters 

3. 1 Position 

3. 2 Injection velocity (trip time) 

3. 3 Injection angle 

3.4 Free return characteristics 


4. 

Planetary Arrival Parameters 


4.1 

Arrival altitude and velocity 


4.2 

Orbit Ephemeris 


4.3 

Place change 


4.4 

Descent trajectory 


4.5 

Hover capability 

5. 

Planetary Departure Parameters 


5.1 

Date 


5.2 

Time 


5.3 

Orbit requirements 


5.4 

Docking maneuvers 


5.5 

De-orbit time and position 


5.6 

Departure velocity requirement 


5.7 

Departure angle 

6. 

Midcourse AV Corrections 


6.1 

Transplanetary requirements 


6.2 

Trans earth requirements 

7. 

Earth Arrival Parameters 


7.1 

Entry time 


7.2 

Entry position 


7.3 

Entry velocity 


7.4 

Entiy angle 


7.5 

Entry altitude 


7.6 

L/D ratio 


7.7 

Landing requirements, site 
locations 

GUIDANCE AND NAVIGATIONAL FACTORS 

1 . 

Launch Vehicle Guidance 


1.1 

Platform errors 


1.2 

Accelerometer accuracy 


1.3 

Computer performance 


1.4 

Vehicle variations 

2. 

Spacecraft Guidance and Navigation 


2.1 

Inertial measuring unit parameters 


2.2 

Optical system performance 


2.3 

Guidance computer performance 


2.4 

Radar parameters 


2.5 

Ground tracking performance 


2.6 

Stabilization and control 
performance 


2.7 

Controls and displays 
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program status. Clearly, the performance data handling problem could be as large or 
larger than weight data processing. The use of computers for performance data proc- 
essing would be essential. 

Data on selected individual items must be reported and monitored on a periodic basis. 
Status and control information would be required for each specific item. Identification 
numbers would be assigned to each item to assure accuracy. 

A typical example of a data reporting format for a rocket engine is shown in Fig- 
ure 6-12. For this figure, the status of key performance parameters is assumed to 
have been requested. Since these engines are used with a variety of oxidizer-to-fuel 
ratios (O/F), specific status is requested for three (say) measured values. For com- 
parison of this specific engine to the class of engines, the average performance curves 
are superimposed on the specific curves. The average performance curves also indi- 
cate the expected standard deviation and the general variation of performance with 
varying O/F ratio. Other data required for the surveillance of engine status is in- 
cluded in Figure 6-12. 

Data request forms similar to Figure 6-12 would be required for each area of per- 
formance considered. While it may be impractical to monitor all significant areas 
of performance, as detailed in Table 6-1, benefits could be derived from surveillance 
of even two or three items. Additional elements could be added as the information and 
control limits became available. Once the flow of status information is established, 
method (b) reduces to a standard application of Forecast Analysis techniques. The 
procedures followed would be quite similar to those being used for weight/performance 
on the Apollo program. 

6.6 APPLICATION TO ELECTRICAL POWER SURVEILLANCE 

The proper application of Forecast Analysis techniques can reduce the adverse impact 
of the electrical load growth on a program. Again, the problem is to select the best 
significant parameters to monitor and to use for predictions of electrical load growth. 
Experience has shown that whenever electrical power parameters, including peak 
power, total energy, required voltage, etc. , are followed on a periodic basis from the 
inception of a program definite patterns of growth are exhibited by the various sys- 
tems. In fact, the patterns are quite consistent from program to program. This leads 
to the selection of the electrical power parameters above for use in Forecast Analysis. 
Measurable requirements are available for these parameters and status measurements 
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SATURN V ROCKET ENGINE PERFORMANCE SUMMARY 
Engine Serial Number Date of This Report 
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Figure 6-12. Typical Engine Performance Chart 





can be made which include Estimated, Calculated, and Actual percentages, as for 
weight data. Once status information flow is established, standard Forecast Analysis 
procedures can be applied to electrical power data. 


Figures 6-13 and 6-14 (taken from "Electrical Power Data Submittal Requirements, " 
CM 006.000-1) show typical input data. The form in Figure 6-13 is designed to sup- 
ply the information necessary to monitor over-all electrical load requirements and 
electrical source capabilities. The form in Figure 6-15 is designed to present the 
data needed to monitor the voltage available at a component. These forms are com- 
pleted for each power source and each electrical load. Figures 6-16 , 6-17, and 6-18 
show typical plots resulting from the application of Forecast Analysis to the input data. 
Figures 6-16 and 6-17 show trend charts for electrical power parameters . An exten- 
sion of a trend to the delivery date graphically shows management if corrective action 
is needed. The minimum load voltage data is presented typically as in Figure 6-18 
in terms of the voltage margin ratio. This ratio is defined as follows: 


VOLTAGE MARGIN RATIO 


MIN. VOLTAGE - MIN. SPEC. VOLTAGE 
MIN. SPEC. VOLTAGE 


Forecast Analysis applies statistical techniques to the well known electrical power 
load growth problem. It provides predictions based on the periodically reported data 
and on the level of program maturity as indicated by the percentage of the reported 
data which is Estimated, Calculated, and Actual. It provides confidence limits on the 
predicted data. It quickly alerts management to any existing or potential problem 
areas and to any changes affecting previous predictions. Management thus has availa- 
ble, at all times, current data and competent predictions on which to base decisions. 
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Power System Mission Profile 



Figure 6-14. Component Voltage Usage 
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NOMENCLATURE AND DEFINITIONS 


There are several terms or nomenclature unique to Forecasts and Appraisals for 
Management Evaluation. These terms are defined below: 


TERM 


DEFINITION 


Buyoff - Required 


Buyoffs- Authorized 


Buyoffs-Potential 


Confidence Limits 


Control Weight 


Deficiency 


E/C/A 


Non-random 
Weight Change 

Normalization 


Mark n 


The actual amount, for example in pounds, which must 
be removed to assure that the control value or limit 
will not be exceeded . 

Weight changes which have officially been approved but 
not formally implemented or incorporated in released 
drawings . 

Includes those changes which have been documented but 
not authorized for incorporation. Can include proposed 
and pending changes. 

Those limits within which a predetermined percentage 
of reported weights will fall for a large number of 
observations. 

The maximum amount of weight permissible for a 
stage , module ,' or spacecraft to be used in a specified 
mission. 

The amount by which a stage, module, or vehicle ex- 
ceeds its control limit, expressible as actual pounds, 
equivalent spacecraft weight, or payload capability. 

Methods by which a parameter, such as weight, is 
determined, with E representing the percent Esti mate d, 
C representing the percent Calculated, and A the per- 
cent Actually measured. 

A change in weight which is not mathematically con- 
sidered as a part of normal weight growth. 

The process of removing the effects of non- random 
weight changes from data prior to trend forecasting. 

A system of computer programs written for the IBM 
7044 Computer, each of which may operate as physi- 
cally independent but functionally consistent units using 
outputs from other programs in the system and infor- 
mation from the Weight Data File. 
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TERM 


DEFINITION 




Model 


Forecast 


Forecast Analysis 


Forecast Line 


Probable Error 


Performance Deficit 
(or Performance 
Variation) 


Repeating Mode 
Analysis 


Targeting 


Mathematical representation of observed behavior, 
used in this manual for forecasting purposes to refer 
to one of four different models; Linear Maximum Like- 
lihood, Non-Linear Maximum Likelihood, Asymptotic 
(Logistic) Exponential, and Adaptive (Fourier) 
Exponential. 

A representation of the sequence of observed and proba- 
ble future weights over a segment of time. 

Forecast Analysis is a process which assesses the 
facts of yesterday, determines the certainties of today, 
and forecasts the probabilistic events of the future. In 
so doing it provides quantitative forecasts of a stated 
condition, (e.g. , weight growth) defines its magnitude, 
and describes the effects of alternate management ac- 
tions or inaction. 

A line extending from the last real data point as dictated 
by the Forecast Analysis model, adjusted by logic, for 
purposes of forecasting a system weight. 

The probable range of a forecast weight. This range is 
expected to be exceeded in no more than one case out 
of 20. 

The amount of equivalent payload weight attributed to 
deviation of performance parameters (e.g. , pro- 
grammed mixture ratio, Isp, etc.) from the set of 
values used to define control weights. 

Basically, a repeating mode program is one which 
analyzes a sequential set of observations (five to six 
points beginning with time zero) and makes forecasts 
for the succeeding months. It then automatically adds 
the next observed point and makes new forecasts for 
succeeding months. This process is repeated until all 
available data is exhausted. A plot is then made of 
these results as a check on the attribute of consistency, 
i.e. , targeting. 

Consistency is that attribute of PAT which is dis- 
tinguished by the convergence of the estimated param- 
eter (in this application weight) toward a final value 
each time an additional set of data is added to the initial 
set of observation. This means that as our knowledge 
improves the probability of forecasting another value, 
other than the one upon which we are converging, 
diminishes rapidly. We choose to refer to this attribute 
as "targeting." 


N-2 



Trade-off Results 


Trend 


The weighing of one alternate result against others to 
arrive at what may be considered the best compromise 
solution possible for a deficiency. 

The direction which weight/ performance appears to 
be taking. 
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